[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:01:32.607834 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:43895
I20260709 13:01:32.609630 13967 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:01:32.610770 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:32.629840 13974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:32.630031 13976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:32.632022 13967 server_base.cc:1061] running on GCE node
W20260709 13:01:32.630276 13973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:33.895117 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:33.895388 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:33.895682 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602093895655 us; error 0 us; skew 500 ppm
I20260709 13:01:33.896559 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:33.902967 13967 webserver.cc:533] Webserver started at http://127.13.163.254:34135/ using document root <none> and password file <none>
I20260709 13:01:33.903971 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:33.904228 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:33.904748 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:33.911674 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "f917f0fa638e4ab681d2c05030192194"
format_stamp: "Formatted at 2026-07-09 13:01:33 on dist-test-slave-w3b1"
I20260709 13:01:33.920258 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:01:33.926235 13982 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:33.927717 13967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:01:33.928283 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "f917f0fa638e4ab681d2c05030192194"
format_stamp: "Formatted at 2026-07-09 13:01:33 on dist-test-slave-w3b1"
I20260709 13:01:33.928680 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:33.984350 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:33.986161 13967 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:01:33.986683 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:34.069134 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:43895
I20260709 13:01:34.069207 14033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:43895 every 8 connection(s)
I20260709 13:01:34.077796 14034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:34.101361 14034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: Bootstrap starting.
I20260709 13:01:34.107690 14034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.109769 14034 log.cc:826] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: Log is configured to *not* fsync() on all Append() calls
I20260709 13:01:34.115064 14034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: No bootstrap required, opened a new log
I20260709 13:01:34.136606 14034 raft_consensus.cc:359] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER }
I20260709 13:01:34.137217 14034 raft_consensus.cc:385] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.137557 14034 raft_consensus.cc:740] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f917f0fa638e4ab681d2c05030192194, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.138607 14034 consensus_queue.cc:260] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER }
I20260709 13:01:34.139156 14034 raft_consensus.cc:399] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:34.139460 14034 raft_consensus.cc:493] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:34.139827 14034 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.144076 14034 raft_consensus.cc:515] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER }
I20260709 13:01:34.144968 14034 leader_election.cc:304] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f917f0fa638e4ab681d2c05030192194; no voters: 
I20260709 13:01:34.146965 14034 leader_election.cc:290] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:34.147347 14037 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.149760 14037 raft_consensus.cc:697] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 1 LEADER]: Becoming Leader. State: Replica: f917f0fa638e4ab681d2c05030192194, State: Running, Role: LEADER
I20260709 13:01:34.150863 14037 consensus_queue.cc:237] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER }
I20260709 13:01:34.152032 14034 sys_catalog.cc:565] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:34.163594 14038 sys_catalog.cc:455] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f917f0fa638e4ab681d2c05030192194. Latest consensus state: current_term: 1 leader_uuid: "f917f0fa638e4ab681d2c05030192194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER } }
I20260709 13:01:34.164393 14038 sys_catalog.cc:458] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:34.165370 14039 sys_catalog.cc:455] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f917f0fa638e4ab681d2c05030192194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f917f0fa638e4ab681d2c05030192194" member_type: VOTER } }
I20260709 13:01:34.166105 14039 sys_catalog.cc:458] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:34.168288 14046 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:34.185622 14046 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:34.188619 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:34.208319 14046 catalog_manager.cc:1383] Generated new cluster ID: fe768bad6d65489e96c7194e4140e0aa
I20260709 13:01:34.208738 14046 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:34.230212 14046 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:34.233237 14046 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:34.250082 14046 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: Generated new TSK 0
I20260709 13:01:34.251209 14046 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:34.320614 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:34.331161 14056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:34.334323 14057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:34.335625 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:34.336712 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:34.337870 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:34.338172 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:34.338398 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602094338378 us; error 0 us; skew 500 ppm
I20260709 13:01:34.339313 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:34.344058 13967 webserver.cc:533] Webserver started at http://127.13.163.193:43109/ using document root <none> and password file <none>
I20260709 13:01:34.344913 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:34.345242 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:34.345628 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:34.347321 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "db2d0dfedce141ce9eb24cd5ad145009"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.355311 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:01:34.361297 14064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.362318 13967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:01:34.362650 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "db2d0dfedce141ce9eb24cd5ad145009"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.362984 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:34.383024 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:34.384478 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:34.387542 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:34.391976 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:34.392267 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.392550 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:34.392761 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.502295 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:41607
I20260709 13:01:34.502398 14126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:41607 every 8 connection(s)
I20260709 13:01:34.510087 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:34.519696 14131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:34.522645 14132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:34.531535 14134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:34.535823 14127 heartbeater.cc:344] Connected to a master server at 127.13.163.254:43895
I20260709 13:01:34.536610 14127 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:34.537060 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:34.537992 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:34.538235 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:34.538113 14127 heartbeater.cc:507] Master 127.13.163.254:43895 requested a full tablet report, sending...
I20260709 13:01:34.538522 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602094538500 us; error 0 us; skew 500 ppm
I20260709 13:01:34.539147 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:34.542480 13999 ts_manager.cc:194] Registered new tserver with Master: db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607)
I20260709 13:01:34.543596 13967 webserver.cc:533] Webserver started at http://127.13.163.194:34213/ using document root <none> and password file <none>
I20260709 13:01:34.544461 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:34.544752 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:34.545130 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:34.545998 13999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37318
I20260709 13:01:34.546797 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "62437b06c9cf4a468c18797b2b40037f"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.553571 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:01:34.558583 14139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.559624 13967 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:01:34.560058 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "62437b06c9cf4a468c18797b2b40037f"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.560495 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:34.600349 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:34.602268 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:34.605181 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:34.609795 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:34.610114 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.610456 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:34.610688 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.704831 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:44289
I20260709 13:01:34.705006 14201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:44289 every 8 connection(s)
I20260709 13:01:34.713969 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:34.726447 14205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:34.728514 14206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:34.730698 14202 heartbeater.cc:344] Connected to a master server at 127.13.163.254:43895
I20260709 13:01:34.731266 14202 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:34.732563 14202 heartbeater.cc:507] Master 127.13.163.254:43895 requested a full tablet report, sending...
I20260709 13:01:34.735781 13999 ts_manager.cc:194] Registered new tserver with Master: 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:34.738051 13999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37324
W20260709 13:01:34.739043 14208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:34.739392 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:34.740873 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:34.741232 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:34.741446 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602094741430 us; error 0 us; skew 500 ppm
I20260709 13:01:34.742182 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:34.745244 13967 webserver.cc:533] Webserver started at http://127.13.163.195:40425/ using document root <none> and password file <none>
I20260709 13:01:34.745887 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:34.746119 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:34.746449 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:34.747907 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "18f6f4c643da436481e453c169625993"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.754293 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:01:34.758697 14213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.759673 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:01:34.760030 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "18f6f4c643da436481e453c169625993"
format_stamp: "Formatted at 2026-07-09 13:01:34 on dist-test-slave-w3b1"
I20260709 13:01:34.760417 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:34.802346 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:34.803741 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:34.805543 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:34.808084 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:34.808352 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.808611 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:34.808804 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:34.910099 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:46611
I20260709 13:01:34.910307 14275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:46611 every 8 connection(s)
I20260709 13:01:34.929580 14276 heartbeater.cc:344] Connected to a master server at 127.13.163.254:43895
I20260709 13:01:34.930070 14276 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:34.931193 14276 heartbeater.cc:507] Master 127.13.163.254:43895 requested a full tablet report, sending...
I20260709 13:01:34.933694 13999 ts_manager.cc:194] Registered new tserver with Master: 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:34.934219 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018770638s
I20260709 13:01:34.935536 13999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37340
I20260709 13:01:34.989578 13999 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37352:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:34.993391 13999 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:35.129983 14167 tablet_service.cc:1511] Processing CreateTablet for tablet 45da4b700b6e4c23b12e26b72a9894ca (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:35.131817 14167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45da4b700b6e4c23b12e26b72a9894ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.132540 14161 tablet_service.cc:1511] Processing CreateTablet for tablet b4530a1c7659435d83bf181a4a5c10be (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:35.134027 14161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4530a1c7659435d83bf181a4a5c10be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.134196 14166 tablet_service.cc:1511] Processing CreateTablet for tablet 108f17b93abd46d8a858cbe195204eed (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:35.135676 14166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108f17b93abd46d8a858cbe195204eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.138226 14165 tablet_service.cc:1511] Processing CreateTablet for tablet f441cf6bf62d422cbac69aadb91af1bd (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:35.139699 14165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f441cf6bf62d422cbac69aadb91af1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.140391 14162 tablet_service.cc:1511] Processing CreateTablet for tablet e08dfd1cdf064abcac5f8219977a26bc (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:35.142778 14162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08dfd1cdf064abcac5f8219977a26bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.152158 14092 tablet_service.cc:1511] Processing CreateTablet for tablet 45da4b700b6e4c23b12e26b72a9894ca (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:35.152029 14091 tablet_service.cc:1511] Processing CreateTablet for tablet 108f17b93abd46d8a858cbe195204eed (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:35.147589 14164 tablet_service.cc:1511] Processing CreateTablet for tablet 346321323ee94d7780a6c723a3989fa0 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:35.148335 14163 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6d25927b6458dbd26f05c291b9482 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:35.155587 14091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108f17b93abd46d8a858cbe195204eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.160800 14090 tablet_service.cc:1511] Processing CreateTablet for tablet f441cf6bf62d422cbac69aadb91af1bd (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:35.167682 14164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346321323ee94d7780a6c723a3989fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.170846 14092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45da4b700b6e4c23b12e26b72a9894ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.173815 14087 tablet_service.cc:1511] Processing CreateTablet for tablet e08dfd1cdf064abcac5f8219977a26bc (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:35.186847 14163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6d25927b6458dbd26f05c291b9482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.167269 14089 tablet_service.cc:1511] Processing CreateTablet for tablet 346321323ee94d7780a6c723a3989fa0 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:35.192337 14241 tablet_service.cc:1511] Processing CreateTablet for tablet 45da4b700b6e4c23b12e26b72a9894ca (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:35.195191 14241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45da4b700b6e4c23b12e26b72a9894ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.194463 14240 tablet_service.cc:1511] Processing CreateTablet for tablet 108f17b93abd46d8a858cbe195204eed (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:35.196514 14088 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6d25927b6458dbd26f05c291b9482 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:35.201411 14090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f441cf6bf62d422cbac69aadb91af1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.219874 14240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108f17b93abd46d8a858cbe195204eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.204957 14086 tablet_service.cc:1511] Processing CreateTablet for tablet b4530a1c7659435d83bf181a4a5c10be (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:35.227401 14239 tablet_service.cc:1511] Processing CreateTablet for tablet f441cf6bf62d422cbac69aadb91af1bd (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:35.224547 14088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6d25927b6458dbd26f05c291b9482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.232086 14085 tablet_service.cc:1511] Processing CreateTablet for tablet 9427a230e89d45d6a386665d5e2c5518 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:35.237088 14089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346321323ee94d7780a6c723a3989fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.233634 14238 tablet_service.cc:1511] Processing CreateTablet for tablet 346321323ee94d7780a6c723a3989fa0 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:35.239041 14086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4530a1c7659435d83bf181a4a5c10be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.240623 14085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9427a230e89d45d6a386665d5e2c5518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.241640 14087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08dfd1cdf064abcac5f8219977a26bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.230075 14239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f441cf6bf62d422cbac69aadb91af1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.230418 14160 tablet_service.cc:1511] Processing CreateTablet for tablet 9427a230e89d45d6a386665d5e2c5518 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:35.243976 14238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346321323ee94d7780a6c723a3989fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.245428 14160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9427a230e89d45d6a386665d5e2c5518. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:35.231514 14047 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:35.294018 14084 tablet_service.cc:1511] Processing CreateTablet for tablet c2d88ef3d5a640dfbf3c8d1685a5afab (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:35.295380 14084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d88ef3d5a640dfbf3c8d1685a5afab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.299121 14237 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6d25927b6458dbd26f05c291b9482 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:35.301113 14237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6d25927b6458dbd26f05c291b9482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.306205 14294 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.310815 14236 tablet_service.cc:1511] Processing CreateTablet for tablet e08dfd1cdf064abcac5f8219977a26bc (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:35.312322 14236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08dfd1cdf064abcac5f8219977a26bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.318626 14295 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.327322 14295 tablet_bootstrap.cc:654] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.329213 14235 tablet_service.cc:1511] Processing CreateTablet for tablet b4530a1c7659435d83bf181a4a5c10be (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:35.332191 14234 tablet_service.cc:1511] Processing CreateTablet for tablet 9427a230e89d45d6a386665d5e2c5518 (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:35.339838 14295 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.340415 14295 ts_tablet_manager.cc:1403] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.009s
I20260709 13:01:35.343832 14295 raft_consensus.cc:359] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.344712 14295 raft_consensus.cc:385] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.345232 14295 raft_consensus.cc:740] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.345563 14235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4530a1c7659435d83bf181a4a5c10be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.346364 14295 consensus_queue.cc:260] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.349467 14234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9427a230e89d45d6a386665d5e2c5518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.376329 14166 tablet_service.cc:1511] Processing CreateTablet for tablet c2d88ef3d5a640dfbf3c8d1685a5afab (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:35.378466 14166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d88ef3d5a640dfbf3c8d1685a5afab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.386204 14294 tablet_bootstrap.cc:654] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.391652 14237 tablet_service.cc:1511] Processing CreateTablet for tablet c2d88ef3d5a640dfbf3c8d1685a5afab (DEFAULT_TABLE table=test-workload [id=85c17bf24351450f86eb4384d76d0ec5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:35.396139 14237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d88ef3d5a640dfbf3c8d1685a5afab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.404515 14295 ts_tablet_manager.cc:1434] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.064s	user 0.022s	sys 0.013s
I20260709 13:01:35.411545 14127 heartbeater.cc:499] Master 127.13.163.254:43895 was elected leader, sending a full tablet report...
I20260709 13:01:35.415148 14295 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.426394 14298 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:35.443282 14294 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:35.443872 14294 ts_tablet_manager.cc:1403] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.138s	user 0.019s	sys 0.000s
I20260709 13:01:35.448891 14295 tablet_bootstrap.cc:654] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.458235 14294 raft_consensus.cc:359] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.459134 14294 raft_consensus.cc:385] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.461907 14294 raft_consensus.cc:740] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.462327 14298 tablet_bootstrap.cc:654] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.464856 14294 consensus_queue.cc:260] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.470865 14298 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:35.471261 14202 heartbeater.cc:499] Master 127.13.163.254:43895 was elected leader, sending a full tablet report...
I20260709 13:01:35.471467 14298 ts_tablet_manager.cc:1403] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.003s
I20260709 13:01:35.472477 14295 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.473026 14295 ts_tablet_manager.cc:1403] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.007s
I20260709 13:01:35.474017 14294 ts_tablet_manager.cc:1434] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:01:35.475493 14294 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.476410 14295 raft_consensus.cc:359] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.477224 14295 raft_consensus.cc:385] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.477581 14295 raft_consensus.cc:740] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.478413 14295 consensus_queue.cc:260] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.480708 14295 ts_tablet_manager.cc:1434] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:35.481765 14295 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.489841 14295 tablet_bootstrap.cc:654] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.491616 14298 raft_consensus.cc:359] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.496300 14294 tablet_bootstrap.cc:654] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.496721 14298 raft_consensus.cc:385] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.500761 14298 raft_consensus.cc:740] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.501858 14298 consensus_queue.cc:260] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.506335 14276 heartbeater.cc:499] Master 127.13.163.254:43895 was elected leader, sending a full tablet report...
I20260709 13:01:35.507272 14298 ts_tablet_manager.cc:1434] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:01:35.508347 14298 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:35.515472 14295 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.516147 14295 ts_tablet_manager.cc:1403] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.024s
W20260709 13:01:35.518666 14128 tablet.cc:2406] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:01:35.519274 14294 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:35.519765 14294 ts_tablet_manager.cc:1403] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.045s	user 0.001s	sys 0.010s
I20260709 13:01:35.519618 14295 raft_consensus.cc:359] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.520435 14295 raft_consensus.cc:385] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.520831 14295 raft_consensus.cc:740] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.521821 14295 consensus_queue.cc:260] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.524237 14295 ts_tablet_manager.cc:1434] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:01:35.525391 14295 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.528237 14298 tablet_bootstrap.cc:654] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.531919 14294 raft_consensus.cc:359] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.532744 14295 tablet_bootstrap.cc:654] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.532765 14294 raft_consensus.cc:385] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.533310 14294 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.534221 14294 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.536829 14297 raft_consensus.cc:493] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.537081 14294 ts_tablet_manager.cc:1434] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 13:01:35.537401 14297 raft_consensus.cc:515] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.538065 14294 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.546123 14297 leader_election.cc:290] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.569741 14301 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.570482 14301 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.571720 14298 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:35.572744 14298 ts_tablet_manager.cc:1403] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.000s
I20260709 13:01:35.574958 14301 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.589046 14294 tablet_bootstrap.cc:654] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.590240 14298 raft_consensus.cc:359] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.593297 14298 raft_consensus.cc:385] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.593740 14298 raft_consensus.cc:740] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.597994 14295 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.602988 14295 ts_tablet_manager.cc:1403] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.039s
I20260709 13:01:35.599921 14298 consensus_queue.cc:260] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.616680 14297 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.619220 14297 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.619840 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
W20260709 13:01:35.624096 14067 leader_election.cc:343] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.625008 14298 ts_tablet_manager.cc:1434] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.002s
I20260709 13:01:35.625463 14297 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.626880 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:35.627380 14295 raft_consensus.cc:359] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.628445 14295 raft_consensus.cc:385] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.628283 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:35.628829 14295 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.627702 14177 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 0.
I20260709 13:01:35.629858 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:35.631695 14298 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:35.638890 14295 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.637825 14067 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
W20260709 13:01:35.644992 14068 leader_election.cc:343] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.645587 14068 leader_election.cc:304] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:35.649973 14297 raft_consensus.cc:2804] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.650470 14297 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.650833 14297 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.653167 14295 ts_tablet_manager.cc:1434] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 13:01:35.654222 14295 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.660642 14298 tablet_bootstrap.cc:654] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.663089 14314 raft_consensus.cc:2749] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.664685 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
W20260709 13:01:35.671778 14143 leader_election.cc:343] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.672423 14295 tablet_bootstrap.cc:654] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.675701 14294 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:35.676283 14294 ts_tablet_manager.cc:1403] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.016s
I20260709 13:01:35.691108 14297 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.689260 14294 raft_consensus.cc:359] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.694429 14294 raft_consensus.cc:385] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.695689 14294 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.696105 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:35.696714 14177 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.696504 14294 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.695403 14297 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.704615 14177 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 1.
W20260709 13:01:35.708068 14068 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:46611, user_credentials={real_user=slave}} blocked reactor thread for 73635us
I20260709 13:01:35.708864 14067 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:35.709093 14294 ts_tablet_manager.cc:1434] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:01:35.710068 14315 raft_consensus.cc:2804] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.710153 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:35.710587 14294 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.711514 14315 raft_consensus.cc:697] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:35.712409 14315 consensus_queue.cc:237] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.714607 14301 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.715394 14068 leader_election.cc:343] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.715080 14301 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
W20260709 13:01:35.716280 14068 leader_election.cc:343] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.717231 14301 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.718744 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:35.720731 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:35.721181 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
W20260709 13:01:35.723879 14143 leader_election.cc:343] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.727957 14298 tablet_bootstrap.cc:492] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:35.728612 14298 ts_tablet_manager.cc:1403] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.020s
I20260709 13:01:35.724634 14101 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 0.
I20260709 13:01:35.729343 14140 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f; no voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:35.731873 14295 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.732367 14295 ts_tablet_manager.cc:1403] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.004s
I20260709 13:01:35.732486 14140 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993
I20260709 13:01:35.734216 14301 raft_consensus.cc:2749] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.739485 14298 raft_consensus.cc:359] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.736034 14301 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.741106 14301 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.741695 14301 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.743197 14298 raft_consensus.cc:385] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.755167 14298 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.757411 14298 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.760383 14294 tablet_bootstrap.cc:654] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.762295 14298 ts_tablet_manager.cc:1434] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:01:35.763365 14298 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:35.760703 14295 raft_consensus.cc:359] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.769259 14295 raft_consensus.cc:385] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.769610 14295 raft_consensus.cc:740] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.768189 14301 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.770550 14295 consensus_queue.cc:260] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.773787 13996 catalog_manager.cc:5697] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 0 to 1, leader changed from <none> to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.778075 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:35.778682 14101 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.783777 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:35.784409 14250 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.781788 14301 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.789065 14295 ts_tablet_manager.cc:1434] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.000s
I20260709 13:01:35.790464 14295 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.795094 14298 tablet_bootstrap.cc:654] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.799049 14101 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:35.801868 14140 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:35.803218 14301 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.803759 14301 raft_consensus.cc:697] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:35.804721 14301 consensus_queue.cc:237] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.805539 14294 tablet_bootstrap.cc:492] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:35.806190 14294 ts_tablet_manager.cc:1403] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.000s
I20260709 13:01:35.811640 14250 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:35.820191 14295 tablet_bootstrap.cc:654] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.829806 14294 raft_consensus.cc:359] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.830926 14294 raft_consensus.cc:385] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.831352 14294 raft_consensus.cc:740] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.843652 13999 catalog_manager.cc:5697] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 0 to 1, leader changed from <none> to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.832413 14294 consensus_queue.cc:260] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.856401 14294 ts_tablet_manager.cc:1434] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 13:01:35.857764 14294 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.858255 14298 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:35.858945 14298 ts_tablet_manager.cc:1403] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.038s
I20260709 13:01:35.863104 14298 raft_consensus.cc:359] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.866520 14295 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.867026 14295 ts_tablet_manager.cc:1403] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.009s
I20260709 13:01:35.864320 14298 raft_consensus.cc:385] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.867612 14294 tablet_bootstrap.cc:654] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.867743 14298 raft_consensus.cc:740] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.870604 14295 raft_consensus.cc:359] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.871425 14295 raft_consensus.cc:385] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.871776 14295 raft_consensus.cc:740] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.875037 14295 consensus_queue.cc:260] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.878363 14295 ts_tablet_manager.cc:1434] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.006s
I20260709 13:01:35.879396 14295 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:35.881882 14294 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:35.882380 14294 ts_tablet_manager.cc:1403] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.010s
I20260709 13:01:35.869774 14298 consensus_queue.cc:260] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.883041 14305 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.883544 14305 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.885582 14294 raft_consensus.cc:359] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.886113 14295 tablet_bootstrap.cc:654] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.886453 14294 raft_consensus.cc:385] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.886837 14294 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.887619 14294 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.895547 14294 ts_tablet_manager.cc:1434] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 13:01:35.896857 14294 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:35.896857 14305 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:35.909307 14298 ts_tablet_manager.cc:1434] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.001s
I20260709 13:01:35.910516 14298 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:35.912307 14294 tablet_bootstrap.cc:654] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.921082 14315 raft_consensus.cc:493] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.921609 14315 raft_consensus.cc:515] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.923883 14315 leader_election.cc:290] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.925486 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:35.926167 14177 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 0.
I20260709 13:01:35.927932 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:35.929497 14067 leader_election.cc:304] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:35.931082 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:35.928565 14250 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 0.
I20260709 13:01:35.935351 14295 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:35.935727 14315 raft_consensus.cc:2804] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.940070 14315 raft_consensus.cc:493] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.940300 14295 ts_tablet_manager.cc:1403] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.007s
I20260709 13:01:35.940488 14315 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.943740 14295 raft_consensus.cc:359] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.936164 14101 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 0.
I20260709 13:01:35.944713 14295 raft_consensus.cc:385] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.945190 14295 raft_consensus.cc:740] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.946795 14214 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:35.947690 14305 raft_consensus.cc:2804] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.948342 14305 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.948504 14301 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.948875 14305 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.949013 14301 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.951864 14301 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.953166 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:35.954377 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
W20260709 13:01:35.955816 14143 leader_election.cc:343] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.951763 14315 raft_consensus.cc:515] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
W20260709 13:01:35.956920 14140 leader_election.cc:343] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.945940 14295 consensus_queue.cc:260] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.961493 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:35.962073 14250 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.957791 14140 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f; no voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:35.963781 14301 raft_consensus.cc:2749] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.963348 14305 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.965638 14298 tablet_bootstrap.cc:654] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.970607 14250 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 1.
I20260709 13:01:35.971976 14068 leader_election.cc:304] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:35.972074 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:35.972958 14101 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.973491 14297 raft_consensus.cc:2804] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.973966 14297 raft_consensus.cc:697] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:35.974743 14297 consensus_queue.cc:237] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:35.959204 14315 leader_election.cc:290] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:35.959908 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:35.981208 14177 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.982640 14295 ts_tablet_manager.cc:1434] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:01:35.983418 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:35.984978 14295 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
W20260709 13:01:35.986552 14216 leader_election.cc:343] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.989607 14177 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 1.
I20260709 13:01:35.992040 14101 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 1.
I20260709 13:01:35.994863 14305 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:35.996119 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
W20260709 13:01:35.998180 14216 leader_election.cc:343] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.998528 13998 catalog_manager.cc:5697] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 0 to 1, leader changed from <none> to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.003518 14214 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.003901 14294 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:36.004516 14294 ts_tablet_manager.cc:1403] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.108s	user 0.049s	sys 0.043s
I20260709 13:01:36.004416 14305 raft_consensus.cc:2804] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.010859 14295 tablet_bootstrap.cc:654] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.013029 14294 raft_consensus.cc:359] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.014235 14294 raft_consensus.cc:385] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.014691 14294 raft_consensus.cc:740] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.015759 14294 consensus_queue.cc:260] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.019222 14298 tablet_bootstrap.cc:492] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:36.019337 14294 ts_tablet_manager.cc:1434] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:01:36.019822 14298 ts_tablet_manager.cc:1403] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.000s
I20260709 13:01:36.020435 14294 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:36.023077 14298 raft_consensus.cc:359] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.023873 14298 raft_consensus.cc:385] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.024194 14298 raft_consensus.cc:740] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.025134 14298 consensus_queue.cc:260] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.026557 14305 raft_consensus.cc:697] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.027603 14305 consensus_queue.cc:237] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.033218 14294 tablet_bootstrap.cc:654] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.034111 14301 raft_consensus.cc:493] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.034691 14301 raft_consensus.cc:515] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.036182 14298 ts_tablet_manager.cc:1434] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 13:01:36.037257 14301 leader_election.cc:290] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.039047 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.039052 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.041477 14101 raft_consensus.cc:2468] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 0.
I20260709 13:01:36.043422 14140 leader_election.cc:304] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.044396 14301 raft_consensus.cc:2804] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.044788 14301 raft_consensus.cc:493] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.045177 14301 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.046030 14294 tablet_bootstrap.cc:492] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:36.046545 14294 ts_tablet_manager.cc:1403] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.011s
W20260709 13:01:36.049788 14143 leader_election.cc:343] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.049889 14294 raft_consensus.cc:359] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.050752 14294 raft_consensus.cc:385] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.051126 14294 raft_consensus.cc:740] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.051605 14295 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:36.052171 14295 ts_tablet_manager.cc:1403] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.014s
I20260709 13:01:36.052095 14294 consensus_queue.cc:260] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.053254 14298 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:36.057677 14295 raft_consensus.cc:359] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.058573 14295 raft_consensus.cc:385] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.058904 14295 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.059891 14295 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.065728 14295 ts_tablet_manager.cc:1434] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:01:36.066095 14294 ts_tablet_manager.cc:1434] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 13:01:36.063910 14301 raft_consensus.cc:515] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.068652 14295 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: Bootstrap starting.
I20260709 13:01:36.073836 13998 catalog_manager.cc:5697] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.074915 14298 tablet_bootstrap.cc:654] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.076066 14294 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:36.082454 14305 raft_consensus.cc:493] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.091843 14305 raft_consensus.cc:515] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.092231 14301 leader_election.cc:290] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.093751 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
W20260709 13:01:36.103621 14143 leader_election.cc:343] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.112751 14319 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.113520 14319 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.137827 14319 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.139701 14294 tablet_bootstrap.cc:654] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.141211 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.141614 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.142828 14101 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.143152 14102 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 0.
I20260709 13:01:36.145771 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.151875 14295 tablet_bootstrap.cc:654] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.158027 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.157315 14140 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.159889 14102 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 0.
I20260709 13:01:36.160673 14298 tablet_bootstrap.cc:492] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:36.161257 14298 ts_tablet_manager.cc:1403] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.108s	user 0.017s	sys 0.013s
I20260709 13:01:36.162204 14250 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.163268 14301 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.158874 14305 leader_election.cc:290] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:36.162360 14143 leader_election.cc:343] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.164458 14301 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.167955 14214 leader_election.cc:304] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.169603 14301 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.170889 14297 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.172345 14305 raft_consensus.cc:2804] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.180610 14294 tablet_bootstrap.cc:492] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:36.182631 14294 ts_tablet_manager.cc:1403] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.111s	user 0.016s	sys 0.005s
I20260709 13:01:36.171043 14298 raft_consensus.cc:359] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.184396 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:36.185150 14298 raft_consensus.cc:385] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.184132 14305 raft_consensus.cc:493] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.187536 14298 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.188179 14305 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.192008 14101 raft_consensus.cc:2468] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:36.192342 14297 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.194397 14301 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.195554 14294 raft_consensus.cc:359] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.196244 14294 raft_consensus.cc:385] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.196677 14140 leader_election.cc:304] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993
I20260709 13:01:36.196532 14294 raft_consensus.cc:740] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.198208 14319 raft_consensus.cc:2804] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.197871 14294 consensus_queue.cc:260] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.198822 14295 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: No bootstrap required, opened a new log
I20260709 13:01:36.199328 14295 ts_tablet_manager.cc:1403] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.006s
I20260709 13:01:36.200189 14305 raft_consensus.cc:515] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.201417 14301 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.206198 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.202906 14319 raft_consensus.cc:697] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.203060 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.202962 14298 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.209139 14319 consensus_queue.cc:237] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.211833 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.212456 14102 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.217656 14251 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.210384 14101 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.220039 14294 ts_tablet_manager.cc:1434] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.037s	user 0.011s	sys 0.008s
I20260709 13:01:36.221329 14294 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: Bootstrap starting.
I20260709 13:01:36.238343 14344 raft_consensus.cc:493] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.238798 14295 raft_consensus.cc:359] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.239418 14295 raft_consensus.cc:385] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.239704 14295 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.240028 14298 ts_tablet_manager.cc:1434] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.007s
I20260709 13:01:36.242041 14298 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:36.242347 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.247504 14251 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:36.249389 14295 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.243000 14305 leader_election.cc:290] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:36.253355 14336 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.253868 14336 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.255707 14294 tablet_bootstrap.cc:654] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:36.255949 14047 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:36.239759 14344 raft_consensus.cc:515] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.259666 14295 ts_tablet_manager.cc:1434] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: Time spent starting tablet: real 0.060s	user 0.001s	sys 0.018s
I20260709 13:01:36.261895 14101 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:36.260418 14177 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.264173 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.264294 14249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.262880 14344 leader_election.cc:290] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.266701 14249 raft_consensus.cc:2393] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 62437b06c9cf4a468c18797b2b40037f in current term 1: Already voted for candidate 18f6f4c643da436481e453c169625993 in this term.
I20260709 13:01:36.266444 14140 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.268616 14140 leader_election.cc:304] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f; no voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.269131 14344 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.270803 14344 raft_consensus.cc:697] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.275555 14102 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 1.
I20260709 13:01:36.277048 14214 leader_election.cc:304] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.278252 14305 raft_consensus.cc:2804] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.278837 14305 raft_consensus.cc:697] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.279850 14305 consensus_queue.cc:237] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.279856 14344 consensus_queue.cc:237] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.284498 14336 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:36.287353 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:36.289798 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.290510 14102 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 1.
I20260709 13:01:36.291767 14214 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.293262 14336 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:36.293660 14336 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.294236 14336 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.296980 14298 tablet_bootstrap.cc:654] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.301893 14345 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.307756 14177 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 1.
I20260709 13:01:36.309059 14319 raft_consensus.cc:2749] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.317534 14294 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: No bootstrap required, opened a new log
I20260709 13:01:36.318377 14294 ts_tablet_manager.cc:1403] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.002s
I20260709 13:01:36.304433 14336 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.323553 14294 raft_consensus.cc:359] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.324601 14294 raft_consensus.cc:385] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.325039 14294 raft_consensus.cc:740] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.326551 14298 tablet_bootstrap.cc:492] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:36.326177 14294 consensus_queue.cc:260] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.327109 14298 ts_tablet_manager.cc:1403] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.011s
I20260709 13:01:36.329035 14294 ts_tablet_manager.cc:1434] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:01:36.315968 14344 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:01:36.336339 14301 raft_consensus.cc:493] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.337041 14301 raft_consensus.cc:515] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.339273 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.340793 14249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.340996 14336 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:36.340909 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.342130 14100 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:36.343331 14143 leader_election.cc:343] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.341876 14301 leader_election.cc:290] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.337968 14298 raft_consensus.cc:359] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.346484 14298 raft_consensus.cc:385] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.346832 14298 raft_consensus.cc:740] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.348017 14298 consensus_queue.cc:260] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.349854 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.350893 14298 ts_tablet_manager.cc:1434] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:01:36.350773 14102 raft_consensus.cc:2468] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 0.
I20260709 13:01:36.351869 14100 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 2.
I20260709 13:01:36.353206 14214 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.354492 14336 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:36.354946 14336 raft_consensus.cc:697] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.355755 14336 consensus_queue.cc:237] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.360961 14298 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:36.357520 14140 leader_election.cc:304] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993
I20260709 13:01:36.365482 14344 raft_consensus.cc:2804] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.365903 14344 raft_consensus.cc:493] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.366338 14344 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.372339 14319 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.372905 14319 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.375756 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.376168 14249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.376564 14102 raft_consensus.cc:2393] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 62437b06c9cf4a468c18797b2b40037f in current term 1: Already voted for candidate 18f6f4c643da436481e453c169625993 in this term.
I20260709 13:01:36.380100 13999 catalog_manager.cc:5697] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 0 to 1, leader changed from <none> to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.377964 14319 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.381582 14140 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f; no voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.382344 14344 raft_consensus.cc:515] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.383821 13999 catalog_manager.cc:5697] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 0 to 1, leader changed from <none> to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.385995 14249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.382704 14319 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.384528 14344 leader_election.cc:290] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
W20260709 13:01:36.387768 14143 leader_election.cc:343] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.389843 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.390949 14100 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.395048 14319 raft_consensus.cc:2749] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.398816 14100 raft_consensus.cc:2468] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 1.
I20260709 13:01:36.400579 14298 tablet_bootstrap.cc:654] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.400419 14140 leader_election.cc:304] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993
I20260709 13:01:36.416467 14314 consensus_queue.cc:1048] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.418321 13997 catalog_manager.cc:5697] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 1 to 2, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.421002 13997 catalog_manager.cc:5697] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.424942 14319 raft_consensus.cc:2804] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.425822 14319 raft_consensus.cc:697] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.426786 14319 consensus_queue.cc:237] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.436668 14298 tablet_bootstrap.cc:492] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:36.437330 14298 ts_tablet_manager.cc:1403] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.022s
I20260709 13:01:36.440212 14298 raft_consensus.cc:359] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.441059 14298 raft_consensus.cc:385] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.441434 14298 raft_consensus.cc:740] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.442215 14298 consensus_queue.cc:260] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.444476 14298 ts_tablet_manager.cc:1434] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:01:36.445798 14298 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: Bootstrap starting.
I20260709 13:01:36.446762 14297 consensus_queue.cc:1048] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.452046 14298 tablet_bootstrap.cc:654] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.464287 14298 tablet_bootstrap.cc:492] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: No bootstrap required, opened a new log
I20260709 13:01:36.464928 14298 ts_tablet_manager.cc:1403] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:01:36.472554 14297 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.473383 14297 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.477821 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:36.478631 14176 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 0.
I20260709 13:01:36.479310 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993" is_pre_election: true
I20260709 13:01:36.479997 14067 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
W20260709 13:01:36.480890 14068 leader_election.cc:343] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18f6f4c643da436481e453c169625993 (127.13.163.195:46611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.481038 14315 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.481451 14315 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.481571 14297 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.481977 14315 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.484998 14298 raft_consensus.cc:359] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.485805 14298 raft_consensus.cc:385] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.486119 14298 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.486877 14298 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.489207 14298 ts_tablet_manager.cc:1434] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:01:36.490536 14315 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.494367 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.495003 14177 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.499269 13998 catalog_manager.cc:5697] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 0 to 1, leader changed from <none> to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.502178 14177 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 1.
I20260709 13:01:36.503544 14067 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.504410 14297 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.505824 14297 raft_consensus.cc:697] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:36.506013 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.506633 14251 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.506996 14297 consensus_queue.cc:237] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.514025 14251 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 1.
I20260709 13:01:36.518829 14315 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 1 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.526139 14336 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.536540 13998 catalog_manager.cc:5697] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 0 to 1, leader changed from <none> to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.547837 14305 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.649472 14251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed"
dest_uuid: "18f6f4c643da436481e453c169625993"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:01:36.650128 14251 raft_consensus.cc:606] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.655701 14251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd"
dest_uuid: "18f6f4c643da436481e453c169625993"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:01:36.656244 14251 raft_consensus.cc:606] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.659929 14100 raft_consensus.cc:1240] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Rejecting Update request from peer 62437b06c9cf4a468c18797b2b40037f for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:36.661693 14319 consensus_queue.cc:1059] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.662719 14319 raft_consensus.cc:3055] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:36.663115 14319 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.664068 14319 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.665783 14319 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.679677 14177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47196
I20260709 13:01:36.680253 14177 raft_consensus.cc:606] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.686354 14365 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.687498 14251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482"
dest_uuid: "18f6f4c643da436481e453c169625993"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:01:36.688118 14251 raft_consensus.cc:606] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.696854 14365 raft_consensus.cc:493] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.697463 14365 raft_consensus.cc:515] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.699592 14344 consensus_queue.cc:1048] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.701412 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62437b06c9cf4a468c18797b2b40037f" is_pre_election: true
I20260709 13:01:36.710963 14101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:41240
I20260709 13:01:36.711539 14101 raft_consensus.cc:606] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.711774 14305 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.719259 14101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:41240
I20260709 13:01:36.719954 14101 raft_consensus.cc:606] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:36.722730 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009" is_pre_election: true
I20260709 13:01:36.723322 14365 leader_election.cc:290] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:36.728127 14214 leader_election.cc:304] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:36.730868 14336 raft_consensus.cc:2749] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.731984 14251 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.732282 14305 raft_consensus.cc:993] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.732772 14305 raft_consensus.cc:1081] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.735021 14177 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47190
I20260709 13:01:36.735685 14177 raft_consensus.cc:493] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.736083 14177 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.738699 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:47196
I20260709 13:01:36.739406 14176 raft_consensus.cc:606] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Received request to transfer leadership to TS 18f6f4c643da436481e453c169625993
I20260709 13:01:36.744568 14345 consensus_queue.cc:1048] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.755854 14177 raft_consensus.cc:515] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.759855 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.760589 14102 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.761478 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.762084 14250 raft_consensus.cc:3055] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:36.762395 14250 raft_consensus.cc:740] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.763247 14250 consensus_queue.cc:260] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.764436 14250 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.768594 14102 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:36.770143 14140 leader_election.cc:304] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.771066 14345 raft_consensus.cc:2804] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:36.771565 14345 raft_consensus.cc:697] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.772419 14345 consensus_queue.cc:237] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.776736 14102 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:36.778594 14365 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.780035 14250 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:36.780967 14177 leader_election.cc:290] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.783385 13998 catalog_manager.cc:5697] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 1 to 2, leader changed from 18f6f4c643da436481e453c169625993 (127.13.163.195) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.793144 14305 raft_consensus.cc:993] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:36.793797 14305 raft_consensus.cc:1081] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:36.798305 14101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41228
I20260709 13:01:36.800112 14177 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:01:36.800668 14177 pending_rounds.cc:85] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Aborting all ops after (but not including) 0
I20260709 13:01:36.800784 14101 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.801355 14101 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.801520 14177 pending_rounds.cc:107] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:01:36.801903 14177 raft_consensus.cc:2889] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.805509 14336 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.811050 14101 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.819295 14101 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 3 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.819195 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.821251 14251 raft_consensus.cc:3055] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:36.821594 14251 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.822538 14251 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.823719 14251 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.819169 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.830511 14177 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.832973 14345 consensus_queue.cc:1048] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.841449 14177 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 3.
I20260709 13:01:36.842950 14067 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.843842 14297 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:36.844340 14297 raft_consensus.cc:697] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:36.845206 14297 consensus_queue.cc:237] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.847026 14251 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 3.
I20260709 13:01:36.857416 14251 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.860725 13998 catalog_manager.cc:5697] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 2 to 3, leader changed from 18f6f4c643da436481e453c169625993 (127.13.163.195) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.864032 14344 consensus_queue.cc:1048] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.874202 14344 raft_consensus.cc:993] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:36.874708 14344 raft_consensus.cc:1081] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Signalling peer 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:36.876595 14251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab"
dest_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:46994
I20260709 13:01:36.877274 14251 raft_consensus.cc:493] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.877681 14251 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.884923 14251 raft_consensus.cc:515] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.886993 14251 leader_election.cc:290] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:36.887861 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.888477 14101 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.888933 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d88ef3d5a640dfbf3c8d1685a5afab" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.889496 14177 raft_consensus.cc:3055] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:36.889832 14177 raft_consensus.cc:740] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.890623 14177 consensus_queue.cc:260] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.891852 14177 raft_consensus.cc:3060] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.895216 14101 raft_consensus.cc:2468] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 2.
I20260709 13:01:36.896333 14214 leader_election.cc:304] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.897116 14336 raft_consensus.cc:2804] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:36.897634 14336 raft_consensus.cc:697] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:36.898479 14336 consensus_queue.cc:237] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.898716 14177 raft_consensus.cc:2468] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 2.
I20260709 13:01:36.906812 14297 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.906188 13996 catalog_manager.cc:5697] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 1 to 2, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.918390 14297 raft_consensus.cc:993] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.918859 14297 raft_consensus.cc:1081] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.921617 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47178
I20260709 13:01:36.922302 14176 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.922713 14176 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.928156 14176 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.930476 14297 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:36.933166 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:36.936122 14176 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.937601 14101 raft_consensus.cc:3055] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:36.938047 14101 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:36.940253 14101 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.941726 14101 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.942050 14249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.949765 14101 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:36.950762 14319 raft_consensus.cc:993] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:36.951012 14140 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 18f6f4c643da436481e453c169625993
I20260709 13:01:36.951263 14319 raft_consensus.cc:1081] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:36.951907 14344 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:36.953219 14344 raft_consensus.cc:697] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.954030 14101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:01:36.954069 14344 consensus_queue.cc:237] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.954952 14101 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.955485 14101 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.960819 14101 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.963587 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:36.964447 14176 raft_consensus.cc:3055] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:36.963587 13998 catalog_manager.cc:5697] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 1 to 2, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.964808 14176 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:36.964705 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:36.965519 14251 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.965741 14176 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.966852 14176 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.972923 14101 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:36.973399 14251 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:36.975601 14068 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:36.976694 14297 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:36.978034 14297 raft_consensus.cc:697] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:36.978012 14176 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:36.979146 14297 consensus_queue.cc:237] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:36.986315 13998 catalog_manager.cc:5697] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 1 to 2, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.993417 14315 raft_consensus.cc:993] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.993904 14315 raft_consensus.cc:1081] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:36.996199 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47178
I20260709 13:01:36.996790 14176 raft_consensus.cc:493] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:36.997287 14176 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.003640 14176 raft_consensus.cc:515] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.005856 14176 leader_election.cc:290] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:37.007020 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:37.007020 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08dfd1cdf064abcac5f8219977a26bc" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:37.007930 14101 raft_consensus.cc:3055] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:37.007966 14251 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.008509 14101 raft_consensus.cc:740] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:37.009519 14101 consensus_queue.cc:260] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.010864 14101 raft_consensus.cc:3060] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.016155 14251 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:37.017602 14143 leader_election.cc:304] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f; no voters: 
I20260709 13:01:37.018543 14344 raft_consensus.cc:2804] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:37.019047 14344 raft_consensus.cc:697] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:37.018914 14101 raft_consensus.cc:2468] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:37.020032 14344 consensus_queue.cc:237] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.027360 13998 catalog_manager.cc:5697] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 1 to 2, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.112102 14365 raft_consensus.cc:993] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:37.112612 14336 raft_consensus.cc:1081] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:37.114382 14101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41228
I20260709 13:01:37.114950 14101 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:37.115334 14101 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.122078 14101 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.124238 14101 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:37.125165 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:37.125614 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:37.125799 14176 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.126224 14251 raft_consensus.cc:3055] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:37.126580 14251 raft_consensus.cc:740] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:37.127454 14251 consensus_queue.cc:260] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.128773 14251 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.133178 14176 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:37.134466 14067 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:37.135296 14315 raft_consensus.cc:2804] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:37.135819 14315 raft_consensus.cc:697] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:37.135872 14251 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:37.136823 14315 consensus_queue.cc:237] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:37.143610 13998 catalog_manager.cc:5697] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 1 to 2, leader changed from 18f6f4c643da436481e453c169625993 (127.13.163.195) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.230973 14101 raft_consensus.cc:1275] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.232709 14344 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.413311 14176 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:37.415231 14249 raft_consensus.cc:1275] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.415356 14102 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.417164 14250 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:37.418516 14344 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.423460 14297 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.424928 14297 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.428318 14102 raft_consensus.cc:1275] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.430799 14365 consensus_queue.cc:1048] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.428710 14319 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:37.439682 14176 raft_consensus.cc:1275] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.441365 14365 consensus_queue.cc:1048] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.486377 14250 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.491818 14250 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.493510 14344 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.525168 14177 raft_consensus.cc:1275] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.526439 14315 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.529374 14101 raft_consensus.cc:1275] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.530836 14344 consensus_queue.cc:1048] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.538650 14250 raft_consensus.cc:1275] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.540237 14383 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.553882 14250 raft_consensus.cc:1275] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.557444 14319 consensus_queue.cc:1048] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.560715 14251 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.563153 14383 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:37.586364 14177 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:37.587888 14314 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:38.359556 14272 debug-util.cc:398] Leaking SignalData structure 0x7b080016e060 after lost signal to thread 13970
W20260709 13:01:38.360291 14272 debug-util.cc:398] Leaking SignalData structure 0x7b080016e740 after lost signal to thread 14033
W20260709 13:01:38.361105 14272 debug-util.cc:398] Leaking SignalData structure 0x7b080017ac20 after lost signal to thread 14126
W20260709 13:01:38.361742 14272 debug-util.cc:398] Leaking SignalData structure 0x7b080017ed00 after lost signal to thread 14201
W20260709 13:01:38.362334 14272 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca400 after lost signal to thread 14275
I20260709 13:01:39.939530 13967 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:01:39.940565 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:39.963579 14100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:41254
I20260709 13:01:39.964166 14100 raft_consensus.cc:606] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Received request to transfer leadership to TS 18f6f4c643da436481e453c169625993
I20260709 13:01:39.975968 14177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:39.976599 14177 raft_consensus.cc:606] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Received request to transfer leadership to TS 18f6f4c643da436481e453c169625993
I20260709 13:01:39.978077 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:39.978574 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:40.152747 14402 raft_consensus.cc:993] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:40.153306 14400 raft_consensus.cc:1081] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Signalling peer 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:40.155232 14250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:40.155897 14250 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:40.156301 14250 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.164211 14250 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.167099 14250 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:40.168232 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:40.168887 14176 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.168992 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:40.169570 14100 raft_consensus.cc:3055] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:40.169802 14100 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:40.170488 14100 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.171505 14100 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.176741 14176 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:40.176960 14100 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:40.178220 14214 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:40.179162 14395 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:40.179694 14395 raft_consensus.cc:697] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:40.180538 14395 consensus_queue.cc:237] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.191668 13997 catalog_manager.cc:5697] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 2 to 3, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.274153 14408 raft_consensus.cc:993] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:40.274688 14381 raft_consensus.cc:1081] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Signalling peer 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:40.276616 14250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:46994
I20260709 13:01:40.277202 14250 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:40.277558 14250 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.282683 14250 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.284528 14250 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:40.285580 14100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:40.285665 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:40.286324 14100 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.286381 14176 raft_consensus.cc:3055] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:40.286929 14176 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:40.287796 14176 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.289180 14176 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.292058 14100 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:40.293402 14214 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:40.294199 14395 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:40.294699 14395 raft_consensus.cc:697] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:40.295488 14395 consensus_queue.cc:237] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:40.297768 14176 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:40.304876 13997 catalog_manager.cc:5697] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 2 to 3, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.714473 14176 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:40.715781 14395 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.737417 14100 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:40.739416 14413 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:40.775602 14176 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:40.777218 14413 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:40.781467 14100 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:40.788946 14395 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:40.979490 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:40.992272 14100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:41240
I20260709 13:01:40.992867 14100 raft_consensus.cc:606] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:40.998349 14100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:41240
I20260709 13:01:40.998950 14100 raft_consensus.cc:606] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:41.005357 14101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:41240
I20260709 13:01:41.006004 14101 raft_consensus.cc:606] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:41.011582 14250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "18f6f4c643da436481e453c169625993"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:01:41.012142 14250 raft_consensus.cc:606] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:41.025198 14250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "18f6f4c643da436481e453c169625993"
mode: GRACEFUL
new_leader_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:01:41.025761 14250 raft_consensus.cc:606] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Received request to transfer leadership to TS 62437b06c9cf4a468c18797b2b40037f
I20260709 13:01:41.047574 14418 raft_consensus.cc:993] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.048121 14400 raft_consensus.cc:1081] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.050056 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47178
I20260709 13:01:41.050658 14176 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:41.051316 14176 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.058411 14176 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.060818 14176 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:41.061268 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:41.061496 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:41.061956 14101 raft_consensus.cc:3055] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:41.062135 14250 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.062330 14101 raft_consensus.cc:740] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:41.063432 14101 consensus_queue.cc:260] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.064797 14101 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.068086 14250 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:41.069547 14143 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f; no voters: 
I20260709 13:01:41.070322 14381 raft_consensus.cc:2804] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:41.071579 14381 raft_consensus.cc:697] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:41.072360 14101 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 2.
I20260709 13:01:41.072525 14381 consensus_queue.cc:237] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.081090 13996 catalog_manager.cc:5697] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 1 to 2, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:41.147761 14418 raft_consensus.cc:993] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.148331 14400 raft_consensus.cc:1081] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.150115 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47178
I20260709 13:01:41.150823 14176 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:41.151268 14176 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.159605 14176 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.162218 14176 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:41.162880 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:41.163465 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108f17b93abd46d8a858cbe195204eed" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:41.164131 14250 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.163506 14101 raft_consensus.cc:3055] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:41.165135 14101 raft_consensus.cc:740] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:41.166158 14101 consensus_queue.cc:260] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.167646 14101 raft_consensus.cc:3060] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.169591 14250 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.170574 14143 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f; no voters: 
I20260709 13:01:41.171255 14381 raft_consensus.cc:2804] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:41.171705 14381 raft_consensus.cc:697] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:41.172508 14381 consensus_queue.cc:237] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.175123 14101 raft_consensus.cc:2468] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.180787 13996 catalog_manager.cc:5697] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 3 to 4, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 4 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:41.217612 14365 raft_consensus.cc:993] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.218189 14395 raft_consensus.cc:1081] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.219874 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47190
I20260709 13:01:41.220572 14176 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:41.221055 14176 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.229658 14176 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.232041 14176 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:41.233229 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:41.233188 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:41.233920 14101 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.233896 14250 raft_consensus.cc:3055] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:41.234504 14250 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:41.235358 14250 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.236486 14250 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.242722 14250 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.243399 14101 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.243986 14143 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f; no voters: 
I20260709 13:01:41.244889 14381 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:41.245429 14381 raft_consensus.cc:697] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:41.246327 14381 consensus_queue.cc:237] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.253687 13996 catalog_manager.cc:5697] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 3 to 4, leader changed from 18f6f4c643da436481e453c169625993 (127.13.163.195) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 4 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:41.370981 14365 raft_consensus.cc:993] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.371498 14395 raft_consensus.cc:1081] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.373298 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47190
I20260709 13:01:41.373867 14176 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:41.374192 14176 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.378738 14176 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.380466 14176 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:41.381430 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:41.381495 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:41.381913 14101 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.382078 14250 raft_consensus.cc:3055] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:41.382416 14250 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:41.382994 14250 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.383852 14250 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.387715 14101 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.388173 14250 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 4.
I20260709 13:01:41.388947 14140 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:41.389689 14381 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:41.390153 14381 raft_consensus.cc:697] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:41.390878 14381 consensus_queue.cc:237] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.397212 13996 catalog_manager.cc:5697] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 3 to 4, leader changed from 18f6f4c643da436481e453c169625993 (127.13.163.195) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 4 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:41.415707 14418 raft_consensus.cc:993] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: : Instructing follower 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.416173 14418 raft_consensus.cc:1081] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Signalling peer 62437b06c9cf4a468c18797b2b40037f to start an election
I20260709 13:01:41.417934 14176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
 from {username='slave'} at 127.0.0.1:47178
I20260709 13:01:41.418445 14176 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:41.418735 14176 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.423298 14176 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.425128 14176 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 3 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:41.426093 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:41.426203 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f441cf6bf62d422cbac69aadb91af1bd" candidate_uuid: "62437b06c9cf4a468c18797b2b40037f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:41.426740 14250 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.426738 14101 raft_consensus.cc:3055] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:41.427253 14101 raft_consensus.cc:740] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:41.427959 14101 consensus_queue.cc:260] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.429056 14101 raft_consensus.cc:3060] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.433116 14250 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 3.
I20260709 13:01:41.434262 14143 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, 62437b06c9cf4a468c18797b2b40037f; no voters: 
I20260709 13:01:41.434737 14101 raft_consensus.cc:2468] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62437b06c9cf4a468c18797b2b40037f in term 3.
I20260709 13:01:41.434937 14381 raft_consensus.cc:2804] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:41.435420 14381 raft_consensus.cc:697] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 3 LEADER]: Becoming Leader. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:41.436163 14381 consensus_queue.cc:237] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:41.442716 13996 catalog_manager.cc:5697] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f reported cstate change: term changed from 2 to 3, leader changed from db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193) to 62437b06c9cf4a468c18797b2b40037f (127.13.163.194). New cstate: current_term: 3 leader_uuid: "62437b06c9cf4a468c18797b2b40037f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:41.561285 14250 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:41.562774 14381 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.568804 14101 raft_consensus.cc:1275] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:41.570402 14381 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:41.577898 14100 raft_consensus.cc:1275] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:41.581817 14381 consensus_queue.cc:1048] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.593180 14250 raft_consensus.cc:1275] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:41.595912 14419 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:41.640368 14100 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:41.641832 14430 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.650264 14250 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:41.651708 14419 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.833717 14100 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:41.834976 14381 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.843479 14250 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:41.845640 14419 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.858417 14100 raft_consensus.cc:1275] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:41.859838 14419 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:41.870402 14250 raft_consensus.cc:1275] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Refusing update from remote peer 62437b06c9cf4a468c18797b2b40037f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:41.871922 14430 consensus_queue.cc:1048] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:44.034725 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:44.041967 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:44.042657 14176 raft_consensus.cc:606] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:44.045917 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:44.046427 14176 raft_consensus.cc:606] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:44.049465 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:44.049935 14176 raft_consensus.cc:606] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Received request to transfer leadership to TS db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:44.052049 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:44.052479 14176 raft_consensus.cc:606] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Received request to transfer leadership to TS 18f6f4c643da436481e453c169625993
I20260709 13:01:44.054881 14176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482"
dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
mode: GRACEFUL
new_leader_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:44.055346 14176 raft_consensus.cc:606] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Received request to transfer leadership to TS 18f6f4c643da436481e453c169625993
I20260709 13:01:44.056391 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:44.056864 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:44.661449 14442 raft_consensus.cc:993] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.662004 14442 raft_consensus.cc:993] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.662398 14442 raft_consensus.cc:1081] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.663391 14442 raft_consensus.cc:1081] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.672977 14102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:01:44.673691 14102 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.674201 14102 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.673905 14100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:01:44.675094 14443 raft_consensus.cc:993] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.675154 14100 raft_consensus.cc:493] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.675657 14443 raft_consensus.cc:1081] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Signalling peer db2d0dfedce141ce9eb24cd5ad145009 to start an election
I20260709 13:01:44.676062 14100 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.677175 14444 raft_consensus.cc:993] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:44.677695 14444 raft_consensus.cc:1081] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Signalling peer 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:44.680747 14250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482"
dest_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:46994
I20260709 13:01:44.681354 14250 raft_consensus.cc:493] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.681705 14250 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.683372 14102 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.686982 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:44.687695 14176 raft_consensus.cc:3055] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:44.687984 14176 raft_consensus.cc:740] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:44.688654 14250 raft_consensus.cc:515] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.688918 14176 consensus_queue.cc:260] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.690073 14176 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.691725 14250 leader_election.cc:290] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:44.691707 14101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:44.692557 14101 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.696938 14101 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:44.698226 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6d25927b6458dbd26f05c291b9482" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:44.698390 14214 leader_election.cc:304] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:44.698920 14177 raft_consensus.cc:3055] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:44.699301 14177 raft_consensus.cc:740] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:44.699383 14395 raft_consensus.cc:2804] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:44.703711 14395 raft_consensus.cc:697] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:44.704798 14395 consensus_queue.cc:237] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.706794 14176 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 5.
I20260709 13:01:44.700285 14177 consensus_queue.cc:260] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.708238 14067 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:44.708670 14177 raft_consensus.cc:3060] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.713984 14100 raft_consensus.cc:515] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.717689 14100 leader_election.cc:290] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:44.717536 14101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca"
dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
 from {username='slave'} at 127.0.0.1:41218
I20260709 13:01:44.718201 14101 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.718539 14101 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.719522 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:44.720063 14176 raft_consensus.cc:3055] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:44.720266 14176 raft_consensus.cc:740] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:44.721004 14176 consensus_queue.cc:260] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.722074 14176 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.722750 14441 raft_consensus.cc:993] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: : Instructing follower 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:44.723217 14441 raft_consensus.cc:1081] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Signalling peer 18f6f4c643da436481e453c169625993 to start an election
I20260709 13:01:44.723289 14102 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 5 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:44.724365 14101 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.725580 14251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be"
dest_uuid: "18f6f4c643da436481e453c169625993"
 from {username='slave'} at 127.0.0.1:46994
I20260709 13:01:44.726172 14251 raft_consensus.cc:493] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:44.726521 14251 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.727967 14177 raft_consensus.cc:2468] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 3.
I20260709 13:01:44.727499 13997 catalog_manager.cc:5697] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 2 to 3, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.730047 14247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9427a230e89d45d6a386665d5e2c5518" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:44.730581 14250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "346321323ee94d7780a6c723a3989fa0" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:44.730638 14247 raft_consensus.cc:3060] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.731153 14250 raft_consensus.cc:3060] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.733700 14251 raft_consensus.cc:515] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.736940 14176 raft_consensus.cc:2468] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:44.737700 14101 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 3 election: Requested vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), 18f6f4c643da436481e453c169625993 (127.13.163.195:46611)
I20260709 13:01:44.738441 14250 raft_consensus.cc:2468] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 5.
I20260709 13:01:44.721405 14448 raft_consensus.cc:2804] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:44.740274 14251 leader_election.cc:290] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 election: Requested vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
I20260709 13:01:44.740382 14177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:44.741106 14177 raft_consensus.cc:3055] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:44.739830 14102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "db2d0dfedce141ce9eb24cd5ad145009"
I20260709 13:01:44.741565 14177 raft_consensus.cc:740] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:44.741955 14102 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.741159 14067 leader_election.cc:304] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:44.742525 14177 consensus_queue.cc:260] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.743765 14177 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.744887 14251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45da4b700b6e4c23b12e26b72a9894ca" candidate_uuid: "db2d0dfedce141ce9eb24cd5ad145009" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18f6f4c643da436481e453c169625993"
I20260709 13:01:44.745594 14251 raft_consensus.cc:3060] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.745910 14450 raft_consensus.cc:2804] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:44.746466 14450 raft_consensus.cc:697] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:44.747429 14450 consensus_queue.cc:237] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.754622 14251 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 3.
I20260709 13:01:44.754622 14176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4530a1c7659435d83bf181a4a5c10be" candidate_uuid: "18f6f4c643da436481e453c169625993" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "62437b06c9cf4a468c18797b2b40037f"
I20260709 13:01:44.755662 14176 raft_consensus.cc:3055] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:44.756102 14176 raft_consensus.cc:740] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 62437b06c9cf4a468c18797b2b40037f, State: Running, Role: LEADER
I20260709 13:01:44.757177 14176 consensus_queue.cc:260] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.761628 14176 raft_consensus.cc:3060] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.761325 14448 raft_consensus.cc:697] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 5 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:44.742935 14247 raft_consensus.cc:2468] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 2.
I20260709 13:01:44.764257 14448 consensus_queue.cc:237] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.770284 14102 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 5.
I20260709 13:01:44.770974 14176 raft_consensus.cc:2468] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18f6f4c643da436481e453c169625993 in term 5.
I20260709 13:01:44.771744 14214 leader_election.cc:304] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:44.773548 14395 raft_consensus.cc:2804] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:44.774112 14395 raft_consensus.cc:697] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 5 LEADER]: Becoming Leader. State: Replica: 18f6f4c643da436481e453c169625993, State: Running, Role: LEADER
I20260709 13:01:44.774262 14177 raft_consensus.cc:2468] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db2d0dfedce141ce9eb24cd5ad145009 in term 3.
I20260709 13:01:44.775131 14395 consensus_queue.cc:237] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.781193 14068 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18f6f4c643da436481e453c169625993, db2d0dfedce141ce9eb24cd5ad145009; no voters: 
I20260709 13:01:44.785096 14450 raft_consensus.cc:2804] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:44.786594 14450 raft_consensus.cc:697] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Becoming Leader. State: Replica: db2d0dfedce141ce9eb24cd5ad145009, State: Running, Role: LEADER
I20260709 13:01:44.788400 14450 consensus_queue.cc:237] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:44.793006 13996 catalog_manager.cc:5697] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 4 to 5, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 5 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:44.802706 13997 catalog_manager.cc:5697] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 reported cstate change: term changed from 4 to 5, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to 18f6f4c643da436481e453c169625993 (127.13.163.195). New cstate: current_term: 5 leader_uuid: "18f6f4c643da436481e453c169625993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.808009 13996 catalog_manager.cc:5697] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 1 to 2, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:44.809221 13996 catalog_manager.cc:5697] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 reported cstate change: term changed from 2 to 3, leader changed from 62437b06c9cf4a468c18797b2b40037f (127.13.163.194) to db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "db2d0dfedce141ce9eb24cd5ad145009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.121987 14102 raft_consensus.cc:1275] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:45.123497 14395 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.133839 14177 raft_consensus.cc:1275] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:45.135535 14446 consensus_queue.cc:1048] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.179510 14177 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:45.181010 14450 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.189772 14247 raft_consensus.cc:1275] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:45.191376 14452 consensus_queue.cc:1048] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.195878 14177 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:45.197331 14446 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:45.207504 14102 raft_consensus.cc:1275] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Refusing update from remote peer 18f6f4c643da436481e453c169625993: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:45.209702 14395 consensus_queue.cc:1048] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:45.237054 14177 raft_consensus.cc:1275] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.237054 14247 raft_consensus.cc:1275] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:45.238669 14448 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.238799 14452 consensus_queue.cc:1048] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.251659 14176 raft_consensus.cc:1275] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:45.253338 14247 raft_consensus.cc:1275] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Refusing update from remote peer db2d0dfedce141ce9eb24cd5ad145009: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:45.253168 14450 consensus_queue.cc:1048] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.254870 14450 consensus_queue.cc:1048] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:45.597745 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:45.604125 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
W20260709 13:01:45.629810 14198 debug-util.cc:398] Leaking SignalData structure 0x7b080012a3e0 after lost signal to thread 13970
W20260709 13:01:45.630813 14198 debug-util.cc:398] Leaking SignalData structure 0x7b080012aac0 after lost signal to thread 14033
W20260709 13:01:45.631917 14198 debug-util.cc:398] Leaking SignalData structure 0x7b08002009a0 after lost signal to thread 14126
W20260709 13:01:45.633284 14198 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8bc0 after lost signal to thread 14201
W20260709 13:01:45.635732 14198 debug-util.cc:398] Leaking SignalData structure 0x7b08001747c0 after lost signal to thread 14275
I20260709 13:01:45.639204 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:45.639976 13967 tablet_replica.cc:333] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.641022 13967 raft_consensus.cc:2243] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.641644 13967 raft_consensus.cc:2272] T e2f6d25927b6458dbd26f05c291b9482 P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.644582 13967 tablet_replica.cc:333] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.645195 13967 raft_consensus.cc:2243] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.645972 13967 raft_consensus.cc:2272] T 9427a230e89d45d6a386665d5e2c5518 P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.648386 13967 tablet_replica.cc:333] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.649070 13967 raft_consensus.cc:2243] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.649523 13967 raft_consensus.cc:2272] T f441cf6bf62d422cbac69aadb91af1bd P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.651331 13967 tablet_replica.cc:333] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.651849 13967 raft_consensus.cc:2243] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.652550 13967 raft_consensus.cc:2272] T 346321323ee94d7780a6c723a3989fa0 P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.654634 13967 tablet_replica.cc:333] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.655135 13967 raft_consensus.cc:2243] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.655565 13967 raft_consensus.cc:2272] T b4530a1c7659435d83bf181a4a5c10be P db2d0dfedce141ce9eb24cd5ad145009 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.657810 13967 tablet_replica.cc:333] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.658371 13967 raft_consensus.cc:2243] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.659090 13967 raft_consensus.cc:2272] T 45da4b700b6e4c23b12e26b72a9894ca P db2d0dfedce141ce9eb24cd5ad145009 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.661072 13967 tablet_replica.cc:333] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.661546 13967 raft_consensus.cc:2243] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.661954 13967 raft_consensus.cc:2272] T e08dfd1cdf064abcac5f8219977a26bc P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.663668 13967 tablet_replica.cc:333] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.664119 13967 raft_consensus.cc:2243] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.664523 13967 raft_consensus.cc:2272] T 108f17b93abd46d8a858cbe195204eed P db2d0dfedce141ce9eb24cd5ad145009 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.666234 13967 tablet_replica.cc:333] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009: stopping tablet replica
I20260709 13:01:45.666675 13967 raft_consensus.cc:2243] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.667054 13967 raft_consensus.cc:2272] T c2d88ef3d5a640dfbf3c8d1685a5afab P db2d0dfedce141ce9eb24cd5ad145009 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.833699 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
W20260709 13:01:45.840668 14214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
W20260709 13:01:45.856802 14214 consensus_peers.cc:597] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 -> Peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Couldn't send request to peer db2d0dfedce141ce9eb24cd5ad145009. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:45.857352 14214 consensus_peers.cc:597] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 -> Peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Couldn't send request to peer db2d0dfedce141ce9eb24cd5ad145009. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:45.867091 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
I20260709 13:01:45.893621 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:45.894284 13967 tablet_replica.cc:333] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.894922 13967 raft_consensus.cc:2243] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.895409 13967 raft_consensus.cc:2272] T e2f6d25927b6458dbd26f05c291b9482 P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.898061 13967 tablet_replica.cc:333] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.898597 13967 raft_consensus.cc:2243] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.899083 13967 raft_consensus.cc:2272] T c2d88ef3d5a640dfbf3c8d1685a5afab P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.900769 13967 tablet_replica.cc:333] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.901312 13967 raft_consensus.cc:2243] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.902064 13967 raft_consensus.cc:2272] T 108f17b93abd46d8a858cbe195204eed P 62437b06c9cf4a468c18797b2b40037f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.903713 13967 tablet_replica.cc:333] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.904220 13967 raft_consensus.cc:2243] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.905045 13967 raft_consensus.cc:2272] T f441cf6bf62d422cbac69aadb91af1bd P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.906658 13967 tablet_replica.cc:333] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.907141 13967 raft_consensus.cc:2243] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.907532 13967 raft_consensus.cc:2272] T 9427a230e89d45d6a386665d5e2c5518 P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.910086 13967 tablet_replica.cc:333] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.910594 13967 raft_consensus.cc:2243] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.910995 13967 raft_consensus.cc:2272] T 346321323ee94d7780a6c723a3989fa0 P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.913229 13967 tablet_replica.cc:333] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.913707 13967 raft_consensus.cc:2243] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.914096 13967 raft_consensus.cc:2272] T 45da4b700b6e4c23b12e26b72a9894ca P 62437b06c9cf4a468c18797b2b40037f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.916262 13967 tablet_replica.cc:333] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.916730 13967 raft_consensus.cc:2243] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.917182 13967 raft_consensus.cc:2272] T b4530a1c7659435d83bf181a4a5c10be P 62437b06c9cf4a468c18797b2b40037f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.919349 13967 tablet_replica.cc:333] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f: stopping tablet replica
I20260709 13:01:45.919812 13967 raft_consensus.cc:2243] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:45.920639 13967 raft_consensus.cc:2272] T e08dfd1cdf064abcac5f8219977a26bc P 62437b06c9cf4a468c18797b2b40037f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:45.985967 14214 consensus_peers.cc:597] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 -> Peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Couldn't send request to peer db2d0dfedce141ce9eb24cd5ad145009. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:46.262058 14216 consensus_peers.cc:597] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 -> Peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Couldn't send request to peer 62437b06c9cf4a468c18797b2b40037f. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:46.288493 14216 consensus_peers.cc:597] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 -> Peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Couldn't send request to peer 62437b06c9cf4a468c18797b2b40037f. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:46.474793 14216 consensus_peers.cc:597] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 -> Peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Couldn't send request to peer 62437b06c9cf4a468c18797b2b40037f. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:46.695258 14473 raft_consensus.cc:493] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader db2d0dfedce141ce9eb24cd5ad145009)
I20260709 13:01:46.695894 14473 raft_consensus.cc:515] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:46.698285 14473 leader_election.cc:290] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:46.704401 14214 leader_election.cc:336] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
W20260709 13:01:46.705054 14216 leader_election.cc:336] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
I20260709 13:01:46.705452 14216 leader_election.cc:304] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:46.706133 14473 raft_consensus.cc:2749] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:46.752481 14473 raft_consensus.cc:493] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db2d0dfedce141ce9eb24cd5ad145009)
I20260709 13:01:46.753057 14473 raft_consensus.cc:515] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:46.754946 14473 leader_election.cc:290] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289), db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607)
I20260709 13:01:46.759297 14473 raft_consensus.cc:493] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db2d0dfedce141ce9eb24cd5ad145009)
W20260709 13:01:46.759898 14216 leader_election.cc:336] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
I20260709 13:01:46.759927 14473 raft_consensus.cc:515] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
W20260709 13:01:46.760624 14214 leader_election.cc:336] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
I20260709 13:01:46.761108 14214 leader_election.cc:304] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:46.761993 14476 raft_consensus.cc:2749] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.762337 14473 leader_election.cc:290] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:46.767308 14214 leader_election.cc:336] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
W20260709 13:01:46.767884 14216 leader_election.cc:336] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
I20260709 13:01:46.768330 14216 leader_election.cc:304] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:46.769002 14473 raft_consensus.cc:2749] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:46.936983 13967 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 14198)
I20260709 13:01:47.020449 14473 raft_consensus.cc:493] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 62437b06c9cf4a468c18797b2b40037f)
I20260709 13:01:47.021037 14473 raft_consensus.cc:515] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:47.022775 14473 leader_election.cc:290] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:47.027398 14216 leader_election.cc:336] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
W20260709 13:01:47.028501 14214 leader_election.cc:336] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
I20260709 13:01:47.028961 14214 leader_election.cc:304] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:47.029599 14473 raft_consensus.cc:2749] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:47.179548 14473 raft_consensus.cc:493] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 62437b06c9cf4a468c18797b2b40037f)
I20260709 13:01:47.180059 14473 raft_consensus.cc:515] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:47.181836 14473 leader_election.cc:290] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:47.186246 14214 leader_election.cc:336] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
W20260709 13:01:47.186790 14216 leader_election.cc:336] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
I20260709 13:01:47.187188 14216 leader_election.cc:304] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:47.187824 14473 raft_consensus.cc:2749] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.270730 14473 raft_consensus.cc:493] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 62437b06c9cf4a468c18797b2b40037f)
I20260709 13:01:47.271237 14473 raft_consensus.cc:515] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2d0dfedce141ce9eb24cd5ad145009" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 41607 } } peers { permanent_uuid: "62437b06c9cf4a468c18797b2b40037f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 44289 } } peers { permanent_uuid: "18f6f4c643da436481e453c169625993" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 46611 } }
I20260709 13:01:47.273028 14473 leader_election.cc:290] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607), 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289)
W20260709 13:01:47.276970 14214 leader_election.cc:336] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer db2d0dfedce141ce9eb24cd5ad145009 (127.13.163.193:41607): Network error: Client connection negotiation failed: client connection to 127.13.163.193:41607: connect: Connection refused (error 111)
W20260709 13:01:47.277595 14216 leader_election.cc:336] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 62437b06c9cf4a468c18797b2b40037f (127.13.163.194:44289): Network error: Client connection negotiation failed: client connection to 127.13.163.194:44289: connect: Connection refused (error 111)
I20260709 13:01:47.277971 14216 leader_election.cc:304] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18f6f4c643da436481e453c169625993; no voters: 62437b06c9cf4a468c18797b2b40037f, db2d0dfedce141ce9eb24cd5ad145009
I20260709 13:01:47.278606 14473 raft_consensus.cc:2749] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:47.834020 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
I20260709 13:01:47.855226 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
I20260709 13:01:47.878608 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:47.879410 13967 tablet_replica.cc:333] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.880046 13967 raft_consensus.cc:2243] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:47.880940 13967 raft_consensus.cc:2272] T c2d88ef3d5a640dfbf3c8d1685a5afab P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.882870 13967 tablet_replica.cc:333] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.883456 13967 raft_consensus.cc:2243] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.883965 13967 raft_consensus.cc:2272] T 108f17b93abd46d8a858cbe195204eed P 18f6f4c643da436481e453c169625993 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.885943 13967 tablet_replica.cc:333] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.886552 13967 raft_consensus.cc:2243] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.886999 13967 raft_consensus.cc:2272] T 9427a230e89d45d6a386665d5e2c5518 P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.888728 13967 tablet_replica.cc:333] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.889319 13967 raft_consensus.cc:2243] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.889803 13967 raft_consensus.cc:2272] T f441cf6bf62d422cbac69aadb91af1bd P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.891640 13967 tablet_replica.cc:333] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.892143 13967 raft_consensus.cc:2243] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.892593 13967 raft_consensus.cc:2272] T 346321323ee94d7780a6c723a3989fa0 P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.894341 13967 tablet_replica.cc:333] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.894869 13967 raft_consensus.cc:2243] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:47.895615 13967 raft_consensus.cc:2272] T e2f6d25927b6458dbd26f05c291b9482 P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.897457 13967 tablet_replica.cc:333] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.898020 13967 raft_consensus.cc:2243] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:47.898794 13967 raft_consensus.cc:2272] T b4530a1c7659435d83bf181a4a5c10be P 18f6f4c643da436481e453c169625993 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.900648 13967 tablet_replica.cc:333] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.901198 13967 raft_consensus.cc:2243] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.901633 13967 raft_consensus.cc:2272] T 45da4b700b6e4c23b12e26b72a9894ca P 18f6f4c643da436481e453c169625993 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.903275 13967 tablet_replica.cc:333] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993: stopping tablet replica
I20260709 13:01:47.904078 13967 raft_consensus.cc:2243] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.904543 13967 raft_consensus.cc:2272] T e08dfd1cdf064abcac5f8219977a26bc P 18f6f4c643da436481e453c169625993 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.954703 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:01:47.972174 13967 master.cc:562] Master@127.13.163.254:43895 shutting down...
I20260709 13:01:48.176611 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:48.177241 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.177603 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P f917f0fa638e4ab681d2c05030192194: stopping tablet replica
I20260709 13:01:48.197793 13967 master.cc:584] Master@127.13.163.254:43895 shutdown complete.
18f6f4c643da436481e453c169625993  1
62437b06c9cf4a468c18797b2b40037f  4
db2d0dfedce141ce9eb24cd5ad145009  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (15634 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:01:48.239825 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:40561
I20260709 13:01:48.241375 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:48.247139 14480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.248539 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.250550 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:48.251258 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:48.252163 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:48.252362 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:48.252498 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602108252485 us; error 0 us; skew 500 ppm
I20260709 13:01:48.253134 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:48.255474 13967 webserver.cc:533] Webserver started at http://127.13.163.254:42659/ using document root <none> and password file <none>
I20260709 13:01:48.255968 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:48.256161 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:48.256444 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:48.257843 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "f1a1dd5e133b415e837864b764847890"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.262149 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:01:48.265494 14488 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.266337 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:01:48.266700 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "f1a1dd5e133b415e837864b764847890"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.266983 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:48.296800 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:48.298097 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:48.337363 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:40561
I20260709 13:01:48.337451 14539 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:40561 every 8 connection(s)
I20260709 13:01:48.341451 14540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.350940 14540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890: Bootstrap starting.
I20260709 13:01:48.355520 14540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:48.360070 14540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890: No bootstrap required, opened a new log
I20260709 13:01:48.362309 14540 raft_consensus.cc:359] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER }
I20260709 13:01:48.362699 14540 raft_consensus.cc:385] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:48.362959 14540 raft_consensus.cc:740] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a1dd5e133b415e837864b764847890, State: Initialized, Role: FOLLOWER
I20260709 13:01:48.363555 14540 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER }
I20260709 13:01:48.364030 14540 raft_consensus.cc:399] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:48.364279 14540 raft_consensus.cc:493] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:48.364583 14540 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:48.368774 14540 raft_consensus.cc:515] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER }
I20260709 13:01:48.369411 14540 leader_election.cc:304] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a1dd5e133b415e837864b764847890; no voters: 
I20260709 13:01:48.370654 14540 leader_election.cc:290] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:48.371271 14543 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:48.372773 14543 raft_consensus.cc:697] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 1 LEADER]: Becoming Leader. State: Replica: f1a1dd5e133b415e837864b764847890, State: Running, Role: LEADER
I20260709 13:01:48.373557 14543 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER }
I20260709 13:01:48.374104 14540 sys_catalog.cc:565] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:48.378479 14544 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1a1dd5e133b415e837864b764847890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER } }
I20260709 13:01:48.379628 14545 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a1dd5e133b415e837864b764847890. Latest consensus state: current_term: 1 leader_uuid: "f1a1dd5e133b415e837864b764847890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a1dd5e133b415e837864b764847890" member_type: VOTER } }
I20260709 13:01:48.380096 14544 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:48.380511 14545 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:48.382648 14551 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:48.391521 14551 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:48.392498 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:48.402781 14551 catalog_manager.cc:1383] Generated new cluster ID: d2e7e924fe8b4cf8b3ec3abf5cb9d616
I20260709 13:01:48.403116 14551 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:48.437886 14551 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:48.439126 14551 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:48.456442 14551 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890: Generated new TSK 0
I20260709 13:01:48.457166 14551 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:48.523703 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:48.530654 14561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.531216 14562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.532558 14564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:48.534024 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:48.534905 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:48.535104 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:48.535236 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602108535224 us; error 0 us; skew 500 ppm
I20260709 13:01:48.535742 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:48.538349 13967 webserver.cc:533] Webserver started at http://127.13.163.193:35721/ using document root <none> and password file <none>
I20260709 13:01:48.538868 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:48.539043 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:48.539278 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:48.540500 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "caeaf499879d4d89a633aafc8d4f1263"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.545388 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:01:48.549011 14569 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.549916 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:01:48.550259 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "caeaf499879d4d89a633aafc8d4f1263"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.550556 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:48.577579 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:48.578872 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:48.580401 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:48.583263 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:48.583535 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.583830 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:48.584044 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.628698 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:36271
I20260709 13:01:48.628875 14631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:36271 every 8 connection(s)
I20260709 13:01:48.633926 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:48.642460 14636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.642964 14637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.646273 14639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:48.646847 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:48.647846 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:48.648054 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:48.648312 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602108648291 us; error 0 us; skew 500 ppm
I20260709 13:01:48.648984 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:48.651338 14632 heartbeater.cc:344] Connected to a master server at 127.13.163.254:40561
I20260709 13:01:48.651690 14632 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:48.652132 13967 webserver.cc:533] Webserver started at http://127.13.163.194:45871/ using document root <none> and password file <none>
I20260709 13:01:48.652704 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:48.652635 14632 heartbeater.cc:507] Master 127.13.163.254:40561 requested a full tablet report, sending...
I20260709 13:01:48.653649 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:48.654040 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:48.655094 14505 ts_manager.cc:194] Registered new tserver with Master: caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271)
I20260709 13:01:48.655298 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.657037 14505 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59678
I20260709 13:01:48.661341 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:01:48.665105 14644 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.665926 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:01:48.666292 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.666626 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:48.680264 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:48.681659 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:48.683177 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:48.685529 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:48.685762 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.686019 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:48.686213 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.729254 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:33511
I20260709 13:01:48.729354 14706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:33511 every 8 connection(s)
I20260709 13:01:48.734354 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:48.742168 14710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:48.744729 14711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:48.746109 13967 server_base.cc:1061] running on GCE node
W20260709 13:01:48.747185 14713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:48.748231 14707 heartbeater.cc:344] Connected to a master server at 127.13.163.254:40561
I20260709 13:01:48.748350 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:48.748682 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:48.748733 14707 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:48.749027 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602108749005 us; error 0 us; skew 500 ppm
I20260709 13:01:48.749713 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:48.749691 14707 heartbeater.cc:507] Master 127.13.163.254:40561 requested a full tablet report, sending...
I20260709 13:01:48.752677 14505 ts_manager.cc:194] Registered new tserver with Master: 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:48.753211 13967 webserver.cc:533] Webserver started at http://127.13.163.195:33175/ using document root <none> and password file <none>
I20260709 13:01:48.753911 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:48.754184 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:48.754519 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:48.754750 14505 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59690
I20260709 13:01:48.756274 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.761381 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:01:48.765242 14718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.766139 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:01:48.766469 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
format_stamp: "Formatted at 2026-07-09 13:01:48 on dist-test-slave-w3b1"
I20260709 13:01:48.766780 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:48.790652 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:48.791898 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:48.793416 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:48.795660 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:48.795894 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.796157 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:48.796344 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:48.837070 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:41891
I20260709 13:01:48.837169 14780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:41891 every 8 connection(s)
I20260709 13:01:48.853868 14781 heartbeater.cc:344] Connected to a master server at 127.13.163.254:40561
I20260709 13:01:48.854301 14781 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:48.855026 14781 heartbeater.cc:507] Master 127.13.163.254:40561 requested a full tablet report, sending...
I20260709 13:01:48.857208 14505 ts_manager.cc:194] Registered new tserver with Master: c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:48.857741 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017356677s
I20260709 13:01:48.858992 14505 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59702
I20260709 13:01:48.886258 14505 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59710:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:48.889175 14505 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:48.959360 14672 tablet_service.cc:1511] Processing CreateTablet for tablet ca11e7cb186347cf82d365cdcee6474f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:48.960945 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca11e7cb186347cf82d365cdcee6474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.964680 14669 tablet_service.cc:1511] Processing CreateTablet for tablet 29a7233db7eb430695d20d08ffeba6b3 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:48.965055 14670 tablet_service.cc:1511] Processing CreateTablet for tablet ad732f811171485182ce493d628ec16a (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:48.966187 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a7233db7eb430695d20d08ffeba6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.966825 14671 tablet_service.cc:1511] Processing CreateTablet for tablet a0fe3f59937449dcb158c31de77239db (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:48.968896 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fe3f59937449dcb158c31de77239db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.970229 14668 tablet_service.cc:1511] Processing CreateTablet for tablet 87fb42e4678a46d08559dd49eeaba53f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:48.971639 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fb42e4678a46d08559dd49eeaba53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.974433 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad732f811171485182ce493d628ec16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.981279 14667 tablet_service.cc:1511] Processing CreateTablet for tablet 150db11da6874fb78006d4ce045cdd23 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:48.984059 14596 tablet_service.cc:1511] Processing CreateTablet for tablet a0fe3f59937449dcb158c31de77239db (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:48.985632 14596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fe3f59937449dcb158c31de77239db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.988071 14666 tablet_service.cc:1511] Processing CreateTablet for tablet 63e7d3ca52644473a44be3dc169dc371 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:48.990191 14597 tablet_service.cc:1511] Processing CreateTablet for tablet ca11e7cb186347cf82d365cdcee6474f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:48.991644 14597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca11e7cb186347cf82d365cdcee6474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.994227 14595 tablet_service.cc:1511] Processing CreateTablet for tablet ad732f811171485182ce493d628ec16a (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:48.995764 14595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad732f811171485182ce493d628ec16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:48.998189 14594 tablet_service.cc:1511] Processing CreateTablet for tablet 29a7233db7eb430695d20d08ffeba6b3 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:48.998984 14590 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca06a8b56a0487594f730031fb98950 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:49.000923 14593 tablet_service.cc:1511] Processing CreateTablet for tablet 87fb42e4678a46d08559dd49eeaba53f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:49.003163 14665 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca06a8b56a0487594f730031fb98950 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:49.003355 14592 tablet_service.cc:1511] Processing CreateTablet for tablet 150db11da6874fb78006d4ce045cdd23 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:49.003902 14745 tablet_service.cc:1511] Processing CreateTablet for tablet a0fe3f59937449dcb158c31de77239db (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:49.005458 14745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0fe3f59937449dcb158c31de77239db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.005925 14591 tablet_service.cc:1511] Processing CreateTablet for tablet 63e7d3ca52644473a44be3dc169dc371 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:49.009207 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150db11da6874fb78006d4ce045cdd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.009070 14589 tablet_service.cc:1511] Processing CreateTablet for tablet 991373c5f7b1425d9a025af65cf249e8 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:49.010179 14744 tablet_service.cc:1511] Processing CreateTablet for tablet ad732f811171485182ce493d628ec16a (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:49.010747 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e7d3ca52644473a44be3dc169dc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.010828 14664 tablet_service.cc:1511] Processing CreateTablet for tablet 991373c5f7b1425d9a025af65cf249e8 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:49.012455 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991373c5f7b1425d9a025af65cf249e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.012198 14743 tablet_service.cc:1511] Processing CreateTablet for tablet 29a7233db7eb430695d20d08ffeba6b3 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:49.013607 14743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a7233db7eb430695d20d08ffeba6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.017258 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca06a8b56a0487594f730031fb98950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.018229 14741 tablet_service.cc:1511] Processing CreateTablet for tablet 150db11da6874fb78006d4ce045cdd23 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:49.019680 14741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150db11da6874fb78006d4ce045cdd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.020642 14594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a7233db7eb430695d20d08ffeba6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.022181 14589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991373c5f7b1425d9a025af65cf249e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.022111 14740 tablet_service.cc:1511] Processing CreateTablet for tablet 63e7d3ca52644473a44be3dc169dc371 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:49.023594 14740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e7d3ca52644473a44be3dc169dc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.024192 14592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150db11da6874fb78006d4ce045cdd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.025465 14742 tablet_service.cc:1511] Processing CreateTablet for tablet 87fb42e4678a46d08559dd49eeaba53f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:49.026706 14746 tablet_service.cc:1511] Processing CreateTablet for tablet ca11e7cb186347cf82d365cdcee6474f (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:49.027750 14590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca06a8b56a0487594f730031fb98950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.028097 14663 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6bfa852bf48908597267c4efa2ee7 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:49.029230 14746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca11e7cb186347cf82d365cdcee6474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.029567 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6bfa852bf48908597267c4efa2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.030889 14744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad732f811171485182ce493d628ec16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.031651 14593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fb42e4678a46d08559dd49eeaba53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.031765 14588 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6bfa852bf48908597267c4efa2ee7 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:49.032794 14742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fb42e4678a46d08559dd49eeaba53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.033499 14588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6bfa852bf48908597267c4efa2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.037231 14591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e7d3ca52644473a44be3dc169dc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.038900 14739 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca06a8b56a0487594f730031fb98950 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:49.040443 14739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca06a8b56a0487594f730031fb98950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.043555 14738 tablet_service.cc:1511] Processing CreateTablet for tablet 991373c5f7b1425d9a025af65cf249e8 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:49.045217 14738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991373c5f7b1425d9a025af65cf249e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.053328 14737 tablet_service.cc:1511] Processing CreateTablet for tablet e2f6bfa852bf48908597267c4efa2ee7 (DEFAULT_TABLE table=test-workload [id=c1611260f15740b2baf8105318c8e72f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:49.054778 14737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f6bfa852bf48908597267c4efa2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.117794 14795 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.141036 14795 tablet_bootstrap.cc:654] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.148474 14796 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.193140 14797 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.200809 14796 tablet_bootstrap.cc:654] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.228253 14797 tablet_bootstrap.cc:654] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.250339 14796 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.250866 14796 ts_tablet_manager.cc:1403] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.004s
I20260709 13:01:49.253973 14796 raft_consensus.cc:359] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.254833 14796 raft_consensus.cc:385] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.255206 14796 raft_consensus.cc:740] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.255700 14795 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.256172 14795 ts_tablet_manager.cc:1403] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.139s	user 0.032s	sys 0.029s
I20260709 13:01:49.256004 14796 consensus_queue.cc:260] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.259163 14795 raft_consensus.cc:359] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.260210 14795 raft_consensus.cc:385] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.260537 14795 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.261490 14795 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.263041 14781 heartbeater.cc:499] Master 127.13.163.254:40561 was elected leader, sending a full tablet report...
I20260709 13:01:49.263767 14796 ts_tablet_manager.cc:1434] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.006s
I20260709 13:01:49.264678 14707 heartbeater.cc:499] Master 127.13.163.254:40561 was elected leader, sending a full tablet report...
I20260709 13:01:49.264945 14796 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.266258 14797 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.266753 14797 ts_tablet_manager.cc:1403] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.012s
I20260709 13:01:49.270128 14797 raft_consensus.cc:359] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.270991 14797 raft_consensus.cc:385] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.271332 14797 raft_consensus.cc:740] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.272058 14796 tablet_bootstrap.cc:654] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.272302 14797 consensus_queue.cc:260] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.273926 14795 ts_tablet_manager.cc:1434] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:01:49.275058 14795 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.287796 14795 tablet_bootstrap.cc:654] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.291740 14632 heartbeater.cc:499] Master 127.13.163.254:40561 was elected leader, sending a full tablet report...
I20260709 13:01:49.292713 14797 ts_tablet_manager.cc:1434] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.026s	user 0.019s	sys 0.008s
I20260709 13:01:49.294487 14796 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.295061 14796 ts_tablet_manager.cc:1403] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.015s
I20260709 13:01:49.295276 14797 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.296489 14801 raft_consensus.cc:493] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.297027 14801 raft_consensus.cc:515] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.298316 14796 raft_consensus.cc:359] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.299186 14796 raft_consensus.cc:385] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.299569 14796 raft_consensus.cc:740] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.300398 14796 consensus_queue.cc:260] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.302893 14797 tablet_bootstrap.cc:654] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.305246 14795 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.305787 14795 ts_tablet_manager.cc:1403] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 13:01:49.306684 14801 leader_election.cc:290] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.310482 14795 raft_consensus.cc:359] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.311435 14795 raft_consensus.cc:385] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.311817 14795 raft_consensus.cc:740] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.312575 14795 consensus_queue.cc:260] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.314822 14795 ts_tablet_manager.cc:1434] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:01:49.319770 14796 ts_tablet_manager.cc:1434] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:01:49.321168 14796 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.326678 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
W20260709 13:01:49.329468 14722 leader_election.cc:343] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.336975 14795 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.344239 14796 tablet_bootstrap.cc:654] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.352246 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.355036 14802 raft_consensus.cc:493] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.355616 14802 raft_consensus.cc:515] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.358513 14795 tablet_bootstrap.cc:654] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:49.359732 14719 leader_election.cc:343] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.360387 14719 leader_election.cc:304] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:49.361408 14806 raft_consensus.cc:2749] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:49.361476 14802 leader_election.cc:290] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.367353 14797 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.367919 14797 ts_tablet_manager.cc:1403] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.073s	user 0.026s	sys 0.023s
I20260709 13:01:49.375531 14797 raft_consensus.cc:359] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.376362 14797 raft_consensus.cc:385] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.376685 14797 raft_consensus.cc:740] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.382328 14796 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.383008 14796 ts_tablet_manager.cc:1403] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.008s
I20260709 13:01:49.382349 14795 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
W20260709 13:01:49.385140 14553 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:49.385555 14795 ts_tablet_manager.cc:1403] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.008s
I20260709 13:01:49.386695 14796 raft_consensus.cc:359] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.387524 14796 raft_consensus.cc:385] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.387881 14796 raft_consensus.cc:740] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.382505 14797 consensus_queue.cc:260] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.388769 14796 consensus_queue.cc:260] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.390878 14797 ts_tablet_manager.cc:1434] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:01:49.391425 14796 ts_tablet_manager.cc:1434] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:49.391862 14797 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.388463 14795 raft_consensus.cc:359] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.392472 14795 raft_consensus.cc:385] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.392542 14796 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.392827 14795 raft_consensus.cc:740] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.393603 14795 consensus_queue.cc:260] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.398660 14797 tablet_bootstrap.cc:654] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.399049 14795 ts_tablet_manager.cc:1434] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:01:49.400105 14795 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.400326 14796 tablet_bootstrap.cc:654] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.407934 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.408629 14795 tablet_bootstrap.cc:654] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.409231 14607 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.411757 14645 leader_election.cc:304] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.413087 14802 raft_consensus.cc:2804] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.413594 14802 raft_consensus.cc:493] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.414047 14802 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.431324 14802 raft_consensus.cc:515] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.435663 14797 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.436201 14797 ts_tablet_manager.cc:1403] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.045s	user 0.027s	sys 0.015s
I20260709 13:01:49.436101 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.436717 14607 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.439342 14797 raft_consensus.cc:359] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.440097 14797 raft_consensus.cc:385] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.440418 14797 raft_consensus.cc:740] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.441332 14797 consensus_queue.cc:260] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.444016 14607 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.445264 14645 leader_election.cc:304] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.448295 14797 ts_tablet_manager.cc:1434] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:01:49.449791 14797 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.457659 14802 leader_election.cc:290] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.458348 14802 raft_consensus.cc:2804] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.458859 14802 raft_consensus.cc:697] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:49.459663 14802 consensus_queue.cc:237] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.463500 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.465662 14797 tablet_bootstrap.cc:654] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.465801 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.466280 14796 tablet_bootstrap.cc:492] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.466543 14795 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.466905 14796 ts_tablet_manager.cc:1403] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.075s	user 0.037s	sys 0.035s
W20260709 13:01:49.467257 14645 leader_election.cc:343] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.466971 14795 ts_tablet_manager.cc:1403] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.002s
W20260709 13:01:49.468358 14645 leader_election.cc:343] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.470402 14796 raft_consensus.cc:359] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.471306 14796 raft_consensus.cc:385] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.471637 14796 raft_consensus.cc:740] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.470566 14795 raft_consensus.cc:359] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.472203 14795 raft_consensus.cc:385] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.472536 14795 raft_consensus.cc:740] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.472357 14796 consensus_queue.cc:260] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.473419 14795 consensus_queue.cc:260] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.478426 14795 ts_tablet_manager.cc:1434] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.005s
I20260709 13:01:49.479444 14795 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.473822 14797 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.480970 14797 ts_tablet_manager.cc:1403] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.003s
I20260709 13:01:49.475399 14796 ts_tablet_manager.cc:1434] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:49.485366 14796 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.484000 14797 raft_consensus.cc:359] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.486671 14797 raft_consensus.cc:385] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.487006 14797 raft_consensus.cc:740] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.487821 14797 consensus_queue.cc:260] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.493167 14795 tablet_bootstrap.cc:654] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.494444 14796 tablet_bootstrap.cc:654] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.497893 14797 ts_tablet_manager.cc:1434] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 13:01:49.504891 14504 catalog_manager.cc:5697] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.507108 14795 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.507835 14795 ts_tablet_manager.cc:1403] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.010s
I20260709 13:01:49.508749 14797 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.511786 14795 raft_consensus.cc:359] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.512569 14795 raft_consensus.cc:385] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.512897 14795 raft_consensus.cc:740] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.513936 14797 tablet_bootstrap.cc:654] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.513818 14795 consensus_queue.cc:260] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.516436 14795 ts_tablet_manager.cc:1434] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:01:49.516589 14796 tablet_bootstrap.cc:492] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.517081 14796 ts_tablet_manager.cc:1403] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.016s
I20260709 13:01:49.517484 14795 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.519955 14796 raft_consensus.cc:359] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.520879 14796 raft_consensus.cc:385] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.521231 14796 raft_consensus.cc:740] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.522156 14796 consensus_queue.cc:260] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.524502 14795 tablet_bootstrap.cc:654] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.525890 14796 ts_tablet_manager.cc:1434] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:49.527482 14796 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.530736 14797 tablet_bootstrap.cc:492] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.531364 14797 ts_tablet_manager.cc:1403] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.023s	user 0.022s	sys 0.000s
I20260709 13:01:49.534694 14796 tablet_bootstrap.cc:654] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.534382 14797 raft_consensus.cc:359] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.535284 14797 raft_consensus.cc:385] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.535677 14797 raft_consensus.cc:740] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.536656 14797 consensus_queue.cc:260] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.542829 14797 ts_tablet_manager.cc:1434] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:01:49.544442 14797 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.550148 14806 raft_consensus.cc:493] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.550922 14806 raft_consensus.cc:515] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.553573 14797 tablet_bootstrap.cc:654] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.556285 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
W20260709 13:01:49.558346 14719 leader_election.cc:343] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.561355 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.561935 14803 raft_consensus.cc:493] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.562649 14803 raft_consensus.cc:515] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
W20260709 13:01:49.563375 14722 leader_election.cc:343] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.564040 14722 leader_election.cc:304] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:49.565352 14801 raft_consensus.cc:2749] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:49.566242 14806 leader_election.cc:290] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.574874 14802 raft_consensus.cc:493] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.575460 14802 raft_consensus.cc:515] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.577720 14802 leader_election.cc:290] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.578850 14803 leader_election.cc:290] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.579540 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.580358 14607 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.581784 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.582700 14756 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.584635 14645 leader_election.cc:304] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.585731 14802 raft_consensus.cc:2804] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.586115 14802 raft_consensus.cc:493] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.586519 14802 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.593806 14802 raft_consensus.cc:515] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.596685 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.597370 14607 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.604627 14607 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.605474 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.605872 14645 leader_election.cc:304] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.606145 14682 raft_consensus.cc:2468] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 0.
I20260709 13:01:49.606858 14816 raft_consensus.cc:2804] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.607509 14573 leader_election.cc:304] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.608188 14802 leader_election.cc:290] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.608420 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.609066 14756 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.608425 14803 raft_consensus.cc:2804] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.612862 14803 raft_consensus.cc:493] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.613265 14803 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.614548 14796 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.614984 14797 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.615067 14796 ts_tablet_manager.cc:1403] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.088s	user 0.033s	sys 0.036s
I20260709 13:01:49.615515 14797 ts_tablet_manager.cc:1403] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.012s
I20260709 13:01:49.618225 14796 raft_consensus.cc:359] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.618481 14797 raft_consensus.cc:359] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.619009 14796 raft_consensus.cc:385] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.618840 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.619443 14797 raft_consensus.cc:385] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.619941 14797 raft_consensus.cc:740] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
W20260709 13:01:49.620888 14570 leader_election.cc:343] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.621050 14795 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.621160 14796 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.621281 14816 raft_consensus.cc:697] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:49.621802 14795 ts_tablet_manager.cc:1403] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.105s	user 0.006s	sys 0.063s
I20260709 13:01:49.620879 14797 consensus_queue.cc:260] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.622138 14796 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.624632 14756 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.624989 14795 raft_consensus.cc:359] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.625743 14795 raft_consensus.cc:385] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.626017 14795 raft_consensus.cc:740] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.622287 14816 consensus_queue.cc:237] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.627559 14803 raft_consensus.cc:515] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.627964 14796 ts_tablet_manager.cc:1434] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:01:49.629280 14796 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.631379 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:49.631950 14682 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.633338 14803 leader_election.cc:290] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.629843 14795 consensus_queue.cc:260] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.634955 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.636497 14795 ts_tablet_manager.cc:1434] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:01:49.637492 14795 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.639331 14830 raft_consensus.cc:493] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.639822 14830 raft_consensus.cc:515] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
W20260709 13:01:49.642846 14570 leader_election.cc:343] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.644215 14797 ts_tablet_manager.cc:1434] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.028s	user 0.015s	sys 0.003s
I20260709 13:01:49.644542 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.641996 14796 tablet_bootstrap.cc:654] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.645282 14607 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.645306 14797 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.648691 14645 leader_election.cc:304] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.649771 14795 tablet_bootstrap.cc:654] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.650383 14802 raft_consensus.cc:2804] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.650764 14802 raft_consensus.cc:493] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.651121 14802 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.653259 14830 leader_election.cc:290] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.655385 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
W20260709 13:01:49.656956 14645 leader_election.cc:343] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.661463 14502 catalog_manager.cc:5697] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.664333 14682 raft_consensus.cc:2468] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 1.
I20260709 13:01:49.665794 14573 leader_election.cc:304] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:49.666970 14831 raft_consensus.cc:2804] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.667486 14831 raft_consensus.cc:697] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:49.668344 14831 consensus_queue.cc:237] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.678706 14802 raft_consensus.cc:515] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.681623 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.682237 14607 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.686321 14797 tablet_bootstrap.cc:654] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.695794 14607 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.697616 14645 leader_election.cc:304] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.698813 14816 raft_consensus.cc:2804] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.699421 14802 leader_election.cc:290] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.699457 14816 raft_consensus.cc:697] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:49.701848 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.701668 14816 consensus_queue.cc:237] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
W20260709 13:01:49.703893 14645 leader_election.cc:343] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.709029 14795 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.709545 14795 ts_tablet_manager.cc:1403] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.072s	user 0.020s	sys 0.037s
I20260709 13:01:49.711480 14797 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.712221 14797 ts_tablet_manager.cc:1403] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.000s
I20260709 13:01:49.712445 14795 raft_consensus.cc:359] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.713415 14795 raft_consensus.cc:385] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.712879 14502 catalog_manager.cc:5697] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 0 to 1, leader changed from <none> to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.713739 14795 raft_consensus.cc:740] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.714960 14795 consensus_queue.cc:260] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.716193 14797 raft_consensus.cc:359] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.719230 14796 tablet_bootstrap.cc:492] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.719743 14796 ts_tablet_manager.cc:1403] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.028s
I20260709 13:01:49.720435 14795 ts_tablet_manager.cc:1434] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:01:49.716883 14797 raft_consensus.cc:385] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.721168 14797 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.721498 14795 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.722214 14797 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.722693 14796 raft_consensus.cc:359] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.723392 14796 raft_consensus.cc:385] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.723737 14796 raft_consensus.cc:740] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.724517 14796 consensus_queue.cc:260] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.725507 14797 ts_tablet_manager.cc:1434] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:01:49.726629 14797 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.726894 14796 ts_tablet_manager.cc:1434] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:49.727509 14816 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.727975 14816 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.730167 14816 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.730338 14796 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.732679 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.733614 14607 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.733891 14797 tablet_bootstrap.cc:654] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.734850 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.735324 14831 raft_consensus.cc:493] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.735846 14831 raft_consensus.cc:515] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
W20260709 13:01:49.736876 14645 leader_election.cc:343] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.737658 14645 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:49.738533 14831 leader_election.cc:290] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.738905 14816 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.739104 14795 tablet_bootstrap.cc:654] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.739295 14816 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.739850 14816 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.740419 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.740422 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
W20260709 13:01:49.743734 14573 leader_election.cc:343] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:49.745188 14570 leader_election.cc:343] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.745818 14570 leader_election.cc:304] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caeaf499879d4d89a633aafc8d4f1263; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:49.747434 14831 raft_consensus.cc:2749] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:49.737664 14796 tablet_bootstrap.cc:654] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.752319 14797 tablet_bootstrap.cc:492] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.734901 14505 catalog_manager.cc:5697] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.753827 14816 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.754508 14797 ts_tablet_manager.cc:1403] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.003s
I20260709 13:01:49.757112 14816 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.758256 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.761591 14607 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.764106 14797 raft_consensus.cc:359] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.765337 14797 raft_consensus.cc:385] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.765821 14797 raft_consensus.cc:740] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.758576 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.767217 14797 consensus_queue.cc:260] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.768772 14607 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.758919 14795 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.770730 14795 ts_tablet_manager.cc:1403] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.017s
I20260709 13:01:49.771550 14797 ts_tablet_manager.cc:1434] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:01:49.773062 14797 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.773597 14795 raft_consensus.cc:359] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.774369 14795 raft_consensus.cc:385] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.774672 14795 raft_consensus.cc:740] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.775573 14795 consensus_queue.cc:260] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.776278 14806 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.776787 14806 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.777911 14795 ts_tablet_manager.cc:1434] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:49.778856 14795 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.778911 14806 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.781212 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.783149 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.783589 14797 tablet_bootstrap.cc:654] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:49.785107 14719 leader_election.cc:343] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.785356 14795 tablet_bootstrap.cc:654] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:49.786350 14722 leader_election.cc:343] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.786856 14722 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:49.788025 14801 raft_consensus.cc:493] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.788666 14801 raft_consensus.cc:515] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.791532 14801 leader_election.cc:290] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.792228 14801 raft_consensus.cc:2749] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:49.797638 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
W20260709 13:01:49.795490 14645 leader_election.cc:343] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:49.798385 14606 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 0.
I20260709 13:01:49.799531 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.800221 14682 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 0.
I20260709 13:01:49.800423 14645 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:49.801764 14722 leader_election.cc:304] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:49.801775 14816 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.804308 14816 raft_consensus.cc:697] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:49.805701 14816 consensus_queue.cc:237] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.803566 14806 raft_consensus.cc:2804] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.810508 14806 raft_consensus.cc:493] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.810943 14806 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.820938 14797 tablet_bootstrap.cc:492] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.821555 14797 ts_tablet_manager.cc:1403] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.049s	user 0.021s	sys 0.002s
I20260709 13:01:49.824295 14797 raft_consensus.cc:359] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.825906 14797 raft_consensus.cc:385] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.826221 14797 raft_consensus.cc:740] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.827097 14796 tablet_bootstrap.cc:492] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.827081 14797 consensus_queue.cc:260] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.827760 14796 ts_tablet_manager.cc:1403] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.098s	user 0.033s	sys 0.024s
I20260709 13:01:49.829377 14797 ts_tablet_manager.cc:1434] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:49.830677 14797 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: Bootstrap starting.
I20260709 13:01:49.830865 14796 raft_consensus.cc:359] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.831385 14802 raft_consensus.cc:493] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.831658 14796 raft_consensus.cc:385] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.832015 14796 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.831895 14802 raft_consensus.cc:515] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.832994 14796 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.834179 14802 leader_election.cc:290] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.834223 14505 catalog_manager.cc:5697] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.836935 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.837656 14607 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.838879 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:49.839622 14756 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:49.841173 14795 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.841642 14795 ts_tablet_manager.cc:1403] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.063s	user 0.020s	sys 0.001s
I20260709 13:01:49.841601 14645 leader_election.cc:304] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:49.842866 14816 raft_consensus.cc:2804] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:49.843601 14816 raft_consensus.cc:493] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.844000 14816 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.844477 14795 raft_consensus.cc:359] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.849823 14795 raft_consensus.cc:385] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.850152 14795 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.850871 14795 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.853608 14796 ts_tablet_manager.cc:1434] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:01:49.842999 14806 raft_consensus.cc:515] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.854732 14796 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.857177 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.857829 14607 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.859011 14797 tablet_bootstrap.cc:654] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.861723 14796 tablet_bootstrap.cc:654] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.863255 14795 ts_tablet_manager.cc:1434] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:01:49.864230 14795 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: Bootstrap starting.
I20260709 13:01:49.865592 14607 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 1.
I20260709 13:01:49.867633 14719 leader_election.cc:304] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:49.868614 14801 raft_consensus.cc:2804] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.867655 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:49.870520 14682 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.872575 14795 tablet_bootstrap.cc:654] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.866536 14806 leader_election.cc:290] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.880739 14816 raft_consensus.cc:515] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.881680 14682 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 1.
I20260709 13:01:49.883054 14801 raft_consensus.cc:697] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:49.884250 14801 consensus_queue.cc:237] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.884598 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:49.885385 14816 leader_election.cc:290] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:49.885633 14756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:49.886201 14756 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.887997 14797 tablet_bootstrap.cc:492] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: No bootstrap required, opened a new log
I20260709 13:01:49.888549 14797 ts_tablet_manager.cc:1403] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: Time spent bootstrapping tablet: real 0.058s	user 0.006s	sys 0.005s
I20260709 13:01:49.890331 14845 raft_consensus.cc:493] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.890878 14845 raft_consensus.cc:515] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.891935 14797 raft_consensus.cc:359] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.892789 14797 raft_consensus.cc:385] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.893095 14797 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.893365 14756 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.893792 14797 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.894163 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.894609 14645 leader_election.cc:304] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:49.895013 14606 raft_consensus.cc:2393] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9cecdd7c2ca4d95a4610ac2bce82947 in current term 1: Already voted for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in this term.
I20260709 13:01:49.895591 14607 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.896610 14816 raft_consensus.cc:2804] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:49.897763 14795 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: No bootstrap required, opened a new log
I20260709 13:01:49.898206 14795 ts_tablet_manager.cc:1403] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.002s
I20260709 13:01:49.899190 14797 ts_tablet_manager.cc:1434] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:01:49.900449 14795 raft_consensus.cc:359] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.901029 14795 raft_consensus.cc:385] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.901232 14795 raft_consensus.cc:740] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.901728 14816 raft_consensus.cc:697] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:49.905158 14816 consensus_queue.cc:237] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.903343 14607 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:49.903968 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.909014 14503 catalog_manager.cc:5697] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.901826 14795 consensus_queue.cc:260] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.906029 14845 leader_election.cc:290] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:49.908663 14756 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.905906 14796 tablet_bootstrap.cc:492] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.910610 14722 leader_election.cc:304] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:49.911059 14796 ts_tablet_manager.cc:1403] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.002s
I20260709 13:01:49.914597 14796 raft_consensus.cc:359] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.915290 14796 raft_consensus.cc:385] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.915549 14796 raft_consensus.cc:740] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.916241 14796 consensus_queue.cc:260] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.918608 14796 ts_tablet_manager.cc:1434] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:49.918952 14795 ts_tablet_manager.cc:1434] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:01:49.918975 14801 raft_consensus.cc:2749] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:49.919538 14796 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: Bootstrap starting.
I20260709 13:01:49.923259 14816 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:49.925390 14796 tablet_bootstrap.cc:654] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:49.927170 14503 catalog_manager.cc:5697] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.948630 14802 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:49.950239 14796 tablet_bootstrap.cc:492] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: No bootstrap required, opened a new log
I20260709 13:01:49.950724 14796 ts_tablet_manager.cc:1403] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.012s
I20260709 13:01:49.953541 14796 raft_consensus.cc:359] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.954248 14796 raft_consensus.cc:385] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:49.954537 14796 raft_consensus.cc:740] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Initialized, Role: FOLLOWER
I20260709 13:01:49.955178 14796 consensus_queue.cc:260] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.957146 14796 ts_tablet_manager.cc:1434] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:01:49.960542 14801 raft_consensus.cc:493] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.961127 14801 raft_consensus.cc:515] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:49.963791 14801 leader_election.cc:290] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271)
I20260709 13:01:49.965158 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:49.966097 14606 raft_consensus.cc:2393] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9cecdd7c2ca4d95a4610ac2bce82947 in current term 1: Already voted for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in this term.
I20260709 13:01:49.966166 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:49.967957 14719 leader_election.cc:304] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:49.968909 14801 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:49.976950 14801 raft_consensus.cc:2749] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:50.036427 14802 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.037052 14802 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.039093 14802 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.040122 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:50.040293 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:50.040809 14755 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:50.040809 14607 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:50.042016 14645 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:50.042747 14802 raft_consensus.cc:2804] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:50.043104 14802 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.043376 14802 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.047978 14802 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.049641 14802 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.050482 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.050791 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.050998 14607 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.051362 14755 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.055879 14607 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:50.056243 14755 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:50.057047 14645 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.057722 14802 raft_consensus.cc:2804] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:50.058109 14802 raft_consensus.cc:697] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.058734 14802 consensus_queue.cc:237] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.066378 14502 catalog_manager.cc:5697] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.084734 14755 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.090839 14801 raft_consensus.cc:493] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.090901 14831 consensus_queue.cc:1048] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:50.091406 14801 raft_consensus.cc:515] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.093281 14801 leader_election.cc:290] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:50.094034 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:50.094329 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:50.094861 14607 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 0.
I20260709 13:01:50.094940 14682 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 0.
I20260709 13:01:50.096279 14719 leader_election.cc:304] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.097167 14801 raft_consensus.cc:2804] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:50.097543 14801 raft_consensus.cc:493] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.097836 14801 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.103482 14801 raft_consensus.cc:515] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.106978 14801 leader_election.cc:290] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:50.108969 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.109694 14681 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.110572 14803 consensus_queue.cc:1048] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.109910 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.114135 14607 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.118479 14681 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 1.
I20260709 13:01:50.120882 14722 leader_election.cc:304] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:50.121667 14607 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 1.
I20260709 13:01:50.122222 14801 raft_consensus.cc:2804] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:50.122982 14801 raft_consensus.cc:697] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:50.124384 14801 consensus_queue.cc:237] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.134152 14503 catalog_manager.cc:5697] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.137403 14802 raft_consensus.cc:493] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.137943 14802 raft_consensus.cc:515] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.139962 14802 leader_election.cc:290] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.141721 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:50.141942 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" is_pre_election: true
I20260709 13:01:50.142385 14607 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:50.142652 14755 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 0.
I20260709 13:01:50.143682 14645 leader_election.cc:304] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.144675 14802 raft_consensus.cc:2804] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:50.145115 14802 raft_consensus.cc:493] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.145473 14802 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.147308 14830 consensus_queue.cc:1048] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.151569 14802 raft_consensus.cc:515] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.154135 14802 leader_election.cc:290] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.155956 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.156632 14755 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.159423 14816 consensus_queue.cc:1048] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.159837 14607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.160465 14607 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.168004 14607 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:50.169720 14816 consensus_queue.cc:1048] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.169894 14645 leader_election.cc:304] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.173481 14755 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 1.
I20260709 13:01:50.173730 14816 raft_consensus.cc:2804] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:50.175527 14816 raft_consensus.cc:697] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.176407 14816 consensus_queue.cc:237] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.186653 14801 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.187275 14801 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.186714 14505 catalog_manager.cc:5697] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 1 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.189338 14801 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:50.190938 14854 consensus_queue.cc:1048] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:50.191882 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" is_pre_election: true
I20260709 13:01:50.193984 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caeaf499879d4d89a633aafc8d4f1263" is_pre_election: true
I20260709 13:01:50.194806 14606 raft_consensus.cc:2393] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9cecdd7c2ca4d95a4610ac2bce82947 in current term 1: Already voted for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in this term.
I20260709 13:01:50.196193 14719 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.197417 14801 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:50.211354 14801 raft_consensus.cc:2749] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:50.218468 14867 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:50.231451 14830 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.365419 14830 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.372984 14801 consensus_queue.cc:1048] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.377724 14802 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.398259 14845 consensus_queue.cc:1048] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.433993 14682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:50.434621 14682 raft_consensus.cc:606] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.440768 14682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:50.441359 14682 raft_consensus.cc:606] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.456697 14755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad732f811171485182ce493d628ec16a"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:53888
I20260709 13:01:50.457382 14755 raft_consensus.cc:606] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.463431 14682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:50.464144 14682 raft_consensus.cc:606] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.470141 14682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:50.470746 14682 raft_consensus.cc:606] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:50.476511 14755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
mode: GRACEFUL
new_leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:53888
I20260709 13:01:50.477207 14755 raft_consensus.cc:606] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Received request to transfer leadership to TS 210fa23e94494ef1b79aac9dc6bd3b9f
I20260709 13:01:50.484266 14830 raft_consensus.cc:993] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.485131 14830 raft_consensus.cc:1081] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.486902 14607 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:50.487583 14607 raft_consensus.cc:493] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.488019 14607 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.495455 14607 raft_consensus.cc:515] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.498540 14607 leader_election.cc:290] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.499095 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.499650 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.499832 14682 raft_consensus.cc:3055] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.500277 14755 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.500320 14682 raft_consensus.cc:740] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.501531 14682 consensus_queue.cc:260] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.502794 14682 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.503146 14607 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:51244
I20260709 13:01:50.503785 14607 raft_consensus.cc:606] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 LEADER]: Received request to transfer leadership to TS 210fa23e94494ef1b79aac9dc6bd3b9f
I20260709 13:01:50.505440 14830 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.511865 14681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:50.512180 14803 raft_consensus.cc:993] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:50.512522 14681 raft_consensus.cc:606] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:50.512699 14803 raft_consensus.cc:1081] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 LEADER]: Signalling peer 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:50.514439 14681 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:46226
I20260709 13:01:50.515098 14681 raft_consensus.cc:493] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.515517 14681 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.514451 14755 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.512002 14682 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.518311 14570 leader_election.cc:304] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.519191 14860 raft_consensus.cc:2804] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.519322 14830 raft_consensus.cc:993] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.519675 14860 raft_consensus.cc:697] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.519887 14830 raft_consensus.cc:1081] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.520543 14860 consensus_queue.cc:237] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.523932 14681 raft_consensus.cc:515] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.524639 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:50.525353 14605 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.525801 14605 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.526347 14681 leader_election.cc:290] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 2 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.527734 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.528411 14755 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.529495 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991373c5f7b1425d9a025af65cf249e8" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.530169 14606 raft_consensus.cc:3055] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.530511 14606 raft_consensus.cc:740] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.531407 14606 consensus_queue.cc:260] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.537807 14755 raft_consensus.cc:2468] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 2.
I20260709 13:01:50.538691 14606 raft_consensus.cc:3060] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.540310 14645 leader_election.cc:304] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:50.540352 14605 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.541054 14867 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.541736 14854 raft_consensus.cc:2804] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.542308 14854 raft_consensus.cc:697] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.543226 14854 consensus_queue.cc:237] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.545285 14606 raft_consensus.cc:2468] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 2.
I20260709 13:01:50.545977 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.546895 14605 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.546618 14681 raft_consensus.cc:3055] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.547950 14681 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.548048 14754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.548635 14754 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.548804 14681 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.548543 14503 catalog_manager.cc:5697] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.549851 14681 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.557034 14681 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.558277 14573 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.559284 14860 raft_consensus.cc:2804] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.558318 14505 catalog_manager.cc:5697] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 1 to 2, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.559756 14860 raft_consensus.cc:697] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.560600 14860 consensus_queue.cc:237] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.561348 14754 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.562726 14755 raft_consensus.cc:1240] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Rejecting Update request from peer 210fa23e94494ef1b79aac9dc6bd3b9f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:01:50.569506 14503 catalog_manager.cc:5697] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.629066 14854 raft_consensus.cc:993] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.629586 14830 raft_consensus.cc:1081] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.631161 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:50.631685 14605 raft_consensus.cc:493] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.632011 14605 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.636768 14605 raft_consensus.cc:515] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.638616 14605 leader_election.cc:290] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.639678 14754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.639556 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.640275 14754 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.640316 14681 raft_consensus.cc:3055] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.640812 14681 raft_consensus.cc:740] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.641651 14681 consensus_queue.cc:260] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.642850 14681 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.647667 14754 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.649155 14570 leader_election.cc:304] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.649892 14681 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.650043 14860 raft_consensus.cc:2804] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.650610 14860 raft_consensus.cc:697] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.651463 14860 consensus_queue.cc:237] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.658326 14503 catalog_manager.cc:5697] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.661875 14845 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.669508 14806 raft_consensus.cc:993] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: : Instructing follower 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:50.670056 14801 raft_consensus.cc:1081] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Signalling peer 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:50.674196 14682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:01:50.674779 14682 raft_consensus.cc:493] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.675190 14682 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.676527 14801 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.685153 14682 raft_consensus.cc:515] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.686918 14682 leader_election.cc:290] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 2 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.688586 14754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.689232 14754 raft_consensus.cc:3055] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.689579 14754 raft_consensus.cc:740] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:50.690678 14754 consensus_queue.cc:260] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.692046 14754 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.695168 14605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.695842 14605 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.698851 14754 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 2.
I20260709 13:01:50.700280 14645 leader_election.cc:304] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:50.701313 14830 raft_consensus.cc:2804] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.701828 14830 raft_consensus.cc:697] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.702998 14605 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 2.
I20260709 13:01:50.703321 14830 consensus_queue.cc:237] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.705155 14802 consensus_queue.cc:1048] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.711732 14505 catalog_manager.cc:5697] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 1 to 2, leader changed from c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195) to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 2 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.717859 14854 consensus_queue.cc:1048] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:50.729597 14816 raft_consensus.cc:993] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:50.730077 14816 raft_consensus.cc:1081] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:50.731947 14754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:53870
I20260709 13:01:50.732681 14754 raft_consensus.cc:493] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.733178 14754 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.739565 14754 raft_consensus.cc:515] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.741472 14754 leader_election.cc:290] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 2 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:50.742548 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:50.742691 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2f6bfa852bf48908597267c4efa2ee7" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.743191 14606 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.743328 14682 raft_consensus.cc:3055] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.743891 14682 raft_consensus.cc:740] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.744791 14682 consensus_queue.cc:260] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.746191 14682 raft_consensus.cc:3060] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.748796 14606 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 2.
I20260709 13:01:50.750008 14719 leader_election.cc:304] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.750710 14801 raft_consensus.cc:2804] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.751166 14801 raft_consensus.cc:697] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:50.752024 14801 consensus_queue.cc:237] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.753103 14682 raft_consensus.cc:2468] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 2.
I20260709 13:01:50.758426 14505 catalog_manager.cc:5697] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.788498 14816 raft_consensus.cc:993] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.789258 14854 raft_consensus.cc:1081] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.790970 14606 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:50.791494 14606 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.791821 14606 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.796475 14606 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.798172 14606 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.799170 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.799398 14754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.799817 14682 raft_consensus.cc:3055] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.799878 14754 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.800279 14682 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:50.801182 14682 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.802420 14682 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.807150 14754 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.808460 14570 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.809065 14682 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.809402 14860 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.809898 14860 raft_consensus.cc:697] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.810770 14860 consensus_queue.cc:237] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.817561 14505 catalog_manager.cc:5697] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.838622 14806 raft_consensus.cc:993] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.839074 14801 raft_consensus.cc:1081] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:50.840545 14606 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad732f811171485182ce493d628ec16a"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:01:50.841135 14606 raft_consensus.cc:493] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.841512 14606 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.846527 14606 raft_consensus.cc:515] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.848371 14606 leader_election.cc:290] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:50.849326 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:50.849583 14754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad732f811171485182ce493d628ec16a" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:50.849869 14682 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.850123 14754 raft_consensus.cc:3055] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:50.850438 14754 raft_consensus.cc:740] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:50.851150 14754 consensus_queue.cc:260] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.852066 14754 raft_consensus.cc:3060] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.855276 14682 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.856781 14573 leader_election.cc:304] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:50.857649 14860 raft_consensus.cc:2804] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:50.858016 14754 raft_consensus.cc:2468] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:50.858170 14860 raft_consensus.cc:697] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:50.859035 14860 consensus_queue.cc:237] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:50.866114 14505 catalog_manager.cc:5697] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.020229 14754 raft_consensus.cc:1275] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.021828 14854 consensus_queue.cc:1048] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.030462 14754 raft_consensus.cc:1275] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.032318 14606 raft_consensus.cc:1275] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.032750 14754 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.032528 14860 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.033986 14854 consensus_queue.cc:1048] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.036234 14682 raft_consensus.cc:1275] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.037724 14859 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.044955 14860 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:51.085640 14681 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.090457 14860 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:51.112977 14755 raft_consensus.cc:1275] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.114405 14867 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.123075 14605 raft_consensus.cc:1275] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.124781 14830 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:51.163223 14755 raft_consensus.cc:1275] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.164640 14860 consensus_queue.cc:1048] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:51.173202 14681 raft_consensus.cc:1275] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.174924 14860 consensus_queue.cc:1048] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.304620 14681 raft_consensus.cc:1275] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.305970 14801 consensus_queue.cc:1048] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.313714 14605 raft_consensus.cc:1275] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.315675 14845 consensus_queue.cc:1048] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.367870 14755 raft_consensus.cc:1275] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.369448 14860 consensus_queue.cc:1048] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.377166 14755 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.379166 14888 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.381623 14681 raft_consensus.cc:1275] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.383116 14888 consensus_queue.cc:1048] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:51.392431 14682 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:51.396045 14888 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:53.548059 13967 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:01:53.549000 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:53.571234 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:53.571954 14606 raft_consensus.cc:606] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Received request to transfer leadership to TS c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:53.574627 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:53.575103 14606 raft_consensus.cc:606] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Received request to transfer leadership to TS c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:53.576102 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:53.576548 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:53.892979 14894 raft_consensus.cc:993] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:53.893584 14860 raft_consensus.cc:1081] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Signalling peer c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:53.895107 14755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:53876
I20260709 13:01:53.895651 14755 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:53.896003 14755 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.901554 14755 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.903512 14755 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:53.904546 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:53.905601 14606 raft_consensus.cc:3055] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:53.905539 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:53.906112 14606 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:53.906396 14681 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.907141 14606 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.908600 14606 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.912703 14681 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:53.914098 14722 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:53.914536 14606 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:53.915205 14845 raft_consensus.cc:2804] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:53.916697 14845 raft_consensus.cc:697] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:53.917728 14845 consensus_queue.cc:237] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.929648 14503 catalog_manager.cc:5697] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 2 to 3, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.959901 14894 raft_consensus.cc:993] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:53.960425 14860 raft_consensus.cc:1081] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Signalling peer c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:53.962069 14755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:53876
I20260709 13:01:53.962582 14755 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:53.962908 14755 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.967394 14755 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.969385 14755 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:53.970145 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:53.970399 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:53.970758 14606 raft_consensus.cc:3055] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:53.971017 14681 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.971127 14606 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:53.972086 14606 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.973277 14606 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.977105 14681 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:53.978380 14722 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:53.978518 14606 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:53.979480 14845 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:53.980006 14845 raft_consensus.cc:697] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:53.981099 14845 consensus_queue.cc:237] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:53.990721 14503 catalog_manager.cc:5697] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 2 to 3, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:54.379781 14681 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:54.381783 14845 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.398415 14606 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:54.400022 14845 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.480322 14605 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:54.481699 14845 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.492560 14681 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:54.493930 14845 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.577474 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:54.591248 14755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:53888
I20260709 13:01:54.591843 14755 raft_consensus.cc:606] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:54.599881 14755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:53888
I20260709 13:01:54.600435 14755 raft_consensus.cc:606] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:54.608085 14681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:54.608640 14681 raft_consensus.cc:606] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:54.613561 14681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:54.614096 14681 raft_consensus.cc:606] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:54.619256 14681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
mode: GRACEFUL
new_leader_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:46232
I20260709 13:01:54.619844 14681 raft_consensus.cc:606] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Received request to transfer leadership to TS caeaf499879d4d89a633aafc8d4f1263
I20260709 13:01:54.670363 14895 raft_consensus.cc:993] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.670934 14908 raft_consensus.cc:1081] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.672420 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:54.672907 14605 raft_consensus.cc:493] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:54.673455 14605 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.679531 14605 raft_consensus.cc:515] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.681253 14605 leader_election.cc:290] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:54.682008 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:54.682348 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca06a8b56a0487594f730031fb98950" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:54.682608 14681 raft_consensus.cc:3055] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:54.682837 14755 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.682972 14681 raft_consensus.cc:740] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:54.683758 14681 consensus_queue.cc:260] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.684784 14681 raft_consensus.cc:3060] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.687418 14755 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:54.688581 14570 leader_election.cc:304] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:54.689442 14894 raft_consensus.cc:2804] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:54.689818 14894 raft_consensus.cc:697] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:54.689873 14681 raft_consensus.cc:2468] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:54.690665 14894 consensus_queue.cc:237] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.698114 14502 catalog_manager.cc:5697] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.789922 14895 raft_consensus.cc:993] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.790432 14895 raft_consensus.cc:1081] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.792141 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:54.792725 14605 raft_consensus.cc:493] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:54.793094 14605 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.798463 14605 raft_consensus.cc:515] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.800567 14605 leader_election.cc:290] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 3 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:54.801470 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:54.801980 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150db11da6874fb78006d4ce045cdd23" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:54.802111 14681 raft_consensus.cc:3055] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:54.802551 14681 raft_consensus.cc:740] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:54.802572 14755 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.803993 14681 consensus_queue.cc:260] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.805069 14681 raft_consensus.cc:3060] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.808202 14755 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 3.
I20260709 13:01:54.810590 14570 leader_election.cc:304] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:54.811116 14681 raft_consensus.cc:2468] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 3.
I20260709 13:01:54.811411 14894 raft_consensus.cc:2804] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:54.811902 14894 raft_consensus.cc:697] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 3 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:54.812870 14894 consensus_queue.cc:237] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.816547 14908 raft_consensus.cc:993] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.817313 14908 raft_consensus.cc:1081] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.819144 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51240
I20260709 13:01:54.819720 14605 raft_consensus.cc:493] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:54.820034 14605 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.821065 14502 catalog_manager.cc:5697] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 2 to 3, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.825817 14605 raft_consensus.cc:515] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.827637 14605 leader_election.cc:290] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:54.828708 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:54.829306 14755 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.829339 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:54.830386 14681 raft_consensus.cc:3055] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:54.830695 14681 raft_consensus.cc:740] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:54.831426 14681 consensus_queue.cc:260] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.832482 14681 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:54.833981 14755 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:54.835078 14570 leader_election.cc:304] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9cecdd7c2ca4d95a4610ac2bce82947, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:54.835949 14894 raft_consensus.cc:2804] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:54.836416 14894 raft_consensus.cc:697] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:54.837391 14894 consensus_queue.cc:237] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:54.838502 14681 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 2.
I20260709 13:01:54.844632 14502 catalog_manager.cc:5697] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 1 to 2, leader changed from 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.996556 14898 raft_consensus.cc:993] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.997182 14845 raft_consensus.cc:1081] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:54.998883 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:01:54.999446 14605 raft_consensus.cc:493] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:54.999809 14605 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.005267 14605 raft_consensus.cc:515] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.007071 14605 leader_election.cc:290] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 4 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:55.008136 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:55.008143 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a7233db7eb430695d20d08ffeba6b3" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:55.008721 14755 raft_consensus.cc:3055] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:55.008802 14681 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.009080 14755 raft_consensus.cc:740] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:55.009779 14755 consensus_queue.cc:260] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.010602 14755 raft_consensus.cc:3060] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.014117 14681 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 4.
I20260709 13:01:55.014979 14755 raft_consensus.cc:2468] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 4.
I20260709 13:01:55.015444 14573 leader_election.cc:304] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:55.016357 14894 raft_consensus.cc:2804] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:55.016752 14894 raft_consensus.cc:697] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:55.017494 14894 consensus_queue.cc:237] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.024163 14502 catalog_manager.cc:5697] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 3 to 4, leader changed from c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 4 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:55.073633 14898 raft_consensus.cc:993] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: : Instructing follower caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:55.074204 14845 raft_consensus.cc:1081] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Signalling peer caeaf499879d4d89a633aafc8d4f1263 to start an election
I20260709 13:01:55.075747 14605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
 from {username='slave'} at 127.0.0.1:51224
I20260709 13:01:55.076238 14605 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:55.076550 14605 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.081467 14605 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.083300 14605 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 4 election: Requested vote from peers 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:55.084197 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:55.084379 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "caeaf499879d4d89a633aafc8d4f1263" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:55.084908 14681 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.084991 14755 raft_consensus.cc:3055] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:55.085398 14755 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:55.086189 14755 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.087486 14755 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.092469 14681 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 4.
I20260709 13:01:55.093786 14573 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, caeaf499879d4d89a633aafc8d4f1263; no voters: 
I20260709 13:01:55.094659 14894 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:55.094812 14755 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caeaf499879d4d89a633aafc8d4f1263 in term 4.
I20260709 13:01:55.095177 14894 raft_consensus.cc:697] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Becoming Leader. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:55.096228 14894 consensus_queue.cc:237] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:55.102453 14502 catalog_manager.cc:5697] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 reported cstate change: term changed from 3 to 4, leader changed from c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195) to caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193). New cstate: current_term: 4 leader_uuid: "caeaf499879d4d89a633aafc8d4f1263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:55.165328 14681 raft_consensus.cc:1275] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:55.166826 14894 consensus_queue.cc:1048] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:55.182778 14755 raft_consensus.cc:1275] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:55.184695 14922 consensus_queue.cc:1048] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:55.198824 14755 raft_consensus.cc:1275] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:55.200098 14922 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:55.212476 14681 raft_consensus.cc:1275] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:55.216037 14894 consensus_queue.cc:1048] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:55.293484 14755 raft_consensus.cc:1275] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:55.294942 14927 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:55.306205 14682 raft_consensus.cc:1275] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:55.309522 14927 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:55.460691 14682 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:55.462189 14927 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:55.472267 14755 raft_consensus.cc:1275] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:55.473937 14927 consensus_queue.cc:1048] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:55.602805 14682 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:55.604394 14927 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:55.613525 14755 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Refusing update from remote peer caeaf499879d4d89a633aafc8d4f1263: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:55.616060 14860 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:57.627903 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:57.635682 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:57.636473 14606 raft_consensus.cc:606] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Received request to transfer leadership to TS 210fa23e94494ef1b79aac9dc6bd3b9f
I20260709 13:01:57.639566 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:57.640322 14606 raft_consensus.cc:606] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Received request to transfer leadership to TS c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:57.642918 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:57.643393 14606 raft_consensus.cc:606] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Received request to transfer leadership to TS c9cecdd7c2ca4d95a4610ac2bce82947
I20260709 13:01:57.645691 14606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f"
dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
mode: GRACEFUL
new_leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:01:57.646231 14606 raft_consensus.cc:606] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Received request to transfer leadership to TS 210fa23e94494ef1b79aac9dc6bd3b9f
I20260709 13:01:57.647204 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:57.647637 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:57.722376 14936 raft_consensus.cc:993] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:57.722971 14927 raft_consensus.cc:1081] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Signalling peer c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:57.724601 14755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:53876
I20260709 13:01:57.725143 14755 raft_consensus.cc:493] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:57.725488 14755 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.729837 14755 raft_consensus.cc:515] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:57.731431 14755 leader_election.cc:290] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 5 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:57.732569 14681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:57.732674 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0fe3f59937449dcb158c31de77239db" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:57.733291 14681 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.733417 14606 raft_consensus.cc:3055] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:57.733856 14606 raft_consensus.cc:740] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:57.734856 14606 consensus_queue.cc:260] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:57.735828 14606 raft_consensus.cc:3060] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.737866 14681 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 5.
I20260709 13:01:57.738425 14682 raft_consensus.cc:1240] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 5 FOLLOWER]: Rejecting Update request from peer caeaf499879d4d89a633aafc8d4f1263 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:57.739104 14722 leader_election.cc:304] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:57.739985 14845 raft_consensus.cc:2804] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:57.740964 14845 raft_consensus.cc:697] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 5 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:57.741794 14845 consensus_queue.cc:237] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:57.742347 14606 raft_consensus.cc:2468] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 5.
I20260709 13:01:57.749351 14502 catalog_manager.cc:5697] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 4 to 5, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 5 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:57.987335 14936 raft_consensus.cc:993] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:57.987805 14936 raft_consensus.cc:1081] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Signalling peer c9cecdd7c2ca4d95a4610ac2bce82947 to start an election
I20260709 13:01:57.989306 14755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f"
dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
 from {username='slave'} at 127.0.0.1:53876
I20260709 13:01:57.989894 14755 raft_consensus.cc:493] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:57.990181 14755 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.994815 14755 raft_consensus.cc:515] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:57.996521 14755 leader_election.cc:290] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511)
I20260709 13:01:57.997154 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:57.997745 14606 raft_consensus.cc:3055] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:57.997805 14682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fb42e4678a46d08559dd49eeaba53f" candidate_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
I20260709 13:01:57.998198 14606 raft_consensus.cc:740] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:57.998427 14682 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.999042 14606 consensus_queue.cc:260] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.000028 14606 raft_consensus.cc:3060] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.003813 14682 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:58.004688 14927 raft_consensus.cc:993] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:58.005110 14722 leader_election.cc:304] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:58.005349 14936 raft_consensus.cc:1081] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Signalling peer 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:58.006004 14845 raft_consensus.cc:2804] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:58.006508 14845 raft_consensus.cc:697] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Becoming Leader. State: Replica: c9cecdd7c2ca4d95a4610ac2bce82947, State: Running, Role: LEADER
I20260709 13:01:58.006938 14606 raft_consensus.cc:2468] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9cecdd7c2ca4d95a4610ac2bce82947 in term 3.
I20260709 13:01:58.007131 14682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:46226
I20260709 13:01:58.007553 14682 raft_consensus.cc:493] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:58.007479 14845 consensus_queue.cc:237] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.007997 14682 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.013311 14682 raft_consensus.cc:515] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.015174 14682 leader_election.cc:290] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 3 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:58.016285 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:58.016811 14606 raft_consensus.cc:3055] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.016098 14502 catalog_manager.cc:5697] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 reported cstate change: term changed from 2 to 3, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.017165 14606 raft_consensus.cc:740] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:58.017758 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e7d3ca52644473a44be3dc169dc371" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:58.018327 14755 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.017897 14606 consensus_queue.cc:260] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.019313 14606 raft_consensus.cc:3060] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.026558 14755 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 3.
I20260709 13:01:58.026558 14606 raft_consensus.cc:2468] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 3.
I20260709 13:01:58.027958 14645 leader_election.cc:304] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:58.028811 14895 raft_consensus.cc:2804] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:58.030045 14895 raft_consensus.cc:697] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:58.030948 14895 consensus_queue.cc:237] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.039988 14502 catalog_manager.cc:5697] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 2 to 3, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 3 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.160136 14682 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 5 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:58.161434 14937 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:58.171028 14606 raft_consensus.cc:1275] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 5 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:58.172874 14948 consensus_queue.cc:1048] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:58.236740 14927 raft_consensus.cc:993] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: : Instructing follower 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:58.237305 14936 raft_consensus.cc:1081] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Signalling peer 210fa23e94494ef1b79aac9dc6bd3b9f to start an election
I20260709 13:01:58.238826 14682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f"
dest_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f"
 from {username='slave'} at 127.0.0.1:46226
I20260709 13:01:58.239302 14682 raft_consensus.cc:493] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:58.239580 14682 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.243819 14682 raft_consensus.cc:515] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.245268 14682 leader_election.cc:290] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 3 election: Requested vote from peers caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271), c9cecdd7c2ca4d95a4610ac2bce82947 (127.13.163.195:41891)
I20260709 13:01:58.246412 14606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "caeaf499879d4d89a633aafc8d4f1263"
I20260709 13:01:58.246472 14755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca11e7cb186347cf82d365cdcee6474f" candidate_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947"
I20260709 13:01:58.246944 14606 raft_consensus.cc:3055] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:58.246966 14755 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.247290 14606 raft_consensus.cc:740] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caeaf499879d4d89a633aafc8d4f1263, State: Running, Role: LEADER
I20260709 13:01:58.247967 14606 consensus_queue.cc:260] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.248811 14606 raft_consensus.cc:3060] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.251744 14755 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 3.
I20260709 13:01:58.252882 14645 leader_election.cc:304] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210fa23e94494ef1b79aac9dc6bd3b9f, c9cecdd7c2ca4d95a4610ac2bce82947; no voters: 
I20260709 13:01:58.253494 14606 raft_consensus.cc:2468] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210fa23e94494ef1b79aac9dc6bd3b9f in term 3.
I20260709 13:01:58.253787 14895 raft_consensus.cc:2804] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:58.254287 14895 raft_consensus.cc:697] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 LEADER]: Becoming Leader. State: Replica: 210fa23e94494ef1b79aac9dc6bd3b9f, State: Running, Role: LEADER
I20260709 13:01:58.255139 14895 consensus_queue.cc:237] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } }
I20260709 13:01:58.262719 14503 catalog_manager.cc:5697] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f reported cstate change: term changed from 2 to 3, leader changed from caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193) to 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194). New cstate: current_term: 3 leader_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.414502 14755 raft_consensus.cc:1275] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:58.415936 14895 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.425107 14606 raft_consensus.cc:1275] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:58.426718 14895 consensus_queue.cc:1048] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.531811 14682 raft_consensus.cc:1275] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:58.533123 14948 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210fa23e94494ef1b79aac9dc6bd3b9f" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.544750 14606 raft_consensus.cc:1275] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Refusing update from remote peer c9cecdd7c2ca4d95a4610ac2bce82947: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:58.546226 14937 consensus_queue.cc:1048] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caeaf499879d4d89a633aafc8d4f1263" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.648442 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:58.655167 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
W20260709 13:01:58.668828 14645 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 35 similar messages]
W20260709 13:01:58.672384 14645 consensus_peers.cc:597] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f -> Peer caeaf499879d4d89a633aafc8d4f1263 (127.13.163.193:36271): Couldn't send request to peer caeaf499879d4d89a633aafc8d4f1263. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:58.685876 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:58.686596 13967 tablet_replica.cc:333] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.687233 13967 raft_consensus.cc:2243] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.687700 13967 raft_consensus.cc:2272] T 63e7d3ca52644473a44be3dc169dc371 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.690652 13967 tablet_replica.cc:333] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.691332 13967 raft_consensus.cc:2243] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.691787 13967 raft_consensus.cc:2272] T a0fe3f59937449dcb158c31de77239db P caeaf499879d4d89a633aafc8d4f1263 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.694247 13967 tablet_replica.cc:333] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.694792 13967 raft_consensus.cc:2243] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.695189 13967 raft_consensus.cc:2272] T ca11e7cb186347cf82d365cdcee6474f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.697130 13967 tablet_replica.cc:333] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.697739 13967 raft_consensus.cc:2243] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.698575 13967 raft_consensus.cc:2272] T ad732f811171485182ce493d628ec16a P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.701094 13967 tablet_replica.cc:333] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.701646 13967 raft_consensus.cc:2243] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.702378 13967 raft_consensus.cc:2272] T 29a7233db7eb430695d20d08ffeba6b3 P caeaf499879d4d89a633aafc8d4f1263 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.704017 13967 tablet_replica.cc:333] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.704515 13967 raft_consensus.cc:2243] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.704969 13967 raft_consensus.cc:2272] T 87fb42e4678a46d08559dd49eeaba53f P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.707058 13967 tablet_replica.cc:333] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.707685 13967 raft_consensus.cc:2243] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.708123 13967 raft_consensus.cc:2272] T e2f6bfa852bf48908597267c4efa2ee7 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.710355 13967 tablet_replica.cc:333] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.711043 13967 raft_consensus.cc:2243] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.712131 13967 raft_consensus.cc:2272] T 2ca06a8b56a0487594f730031fb98950 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.713981 13967 tablet_replica.cc:333] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.714396 13967 raft_consensus.cc:2243] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.715082 13967 raft_consensus.cc:2272] T 150db11da6874fb78006d4ce045cdd23 P caeaf499879d4d89a633aafc8d4f1263 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.716665 13967 tablet_replica.cc:333] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263: stopping tablet replica
I20260709 13:01:58.717156 13967 raft_consensus.cc:2243] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.717590 13967 raft_consensus.cc:2272] T 991373c5f7b1425d9a025af65cf249e8 P caeaf499879d4d89a633aafc8d4f1263 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.748287 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
I20260709 13:01:58.771008 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
I20260709 13:01:58.796994 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:58.797694 13967 tablet_replica.cc:333] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.798305 13967 raft_consensus.cc:2243] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.798753 13967 raft_consensus.cc:2272] T 150db11da6874fb78006d4ce045cdd23 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.800537 13967 tablet_replica.cc:333] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.801158 13967 raft_consensus.cc:2243] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.801878 13967 raft_consensus.cc:2272] T 991373c5f7b1425d9a025af65cf249e8 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.803925 13967 tablet_replica.cc:333] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.804476 13967 raft_consensus.cc:2243] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.804970 13967 raft_consensus.cc:2272] T e2f6bfa852bf48908597267c4efa2ee7 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.806635 13967 tablet_replica.cc:333] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.807122 13967 raft_consensus.cc:2243] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.807579 13967 raft_consensus.cc:2272] T 2ca06a8b56a0487594f730031fb98950 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.809564 13967 tablet_replica.cc:333] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.809490 14754 raft_consensus.cc:1275] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Refusing update from remote peer 210fa23e94494ef1b79aac9dc6bd3b9f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:58.810170 13967 raft_consensus.cc:2243] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.810768 13967 raft_consensus.cc:2272] T 29a7233db7eb430695d20d08ffeba6b3 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.811025 14944 consensus_queue.cc:1048] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cecdd7c2ca4d95a4610ac2bce82947" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.812992 13967 tablet_replica.cc:333] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.813555 13967 raft_consensus.cc:2243] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.814064 13967 raft_consensus.cc:2272] T 87fb42e4678a46d08559dd49eeaba53f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.817056 13967 tablet_replica.cc:333] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.817801 13967 raft_consensus.cc:2243] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.819005 13967 raft_consensus.cc:2272] T ca11e7cb186347cf82d365cdcee6474f P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.821735 13967 tablet_replica.cc:333] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.822257 13967 raft_consensus.cc:2243] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.822716 13967 raft_consensus.cc:2272] T ad732f811171485182ce493d628ec16a P 210fa23e94494ef1b79aac9dc6bd3b9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.824322 13967 tablet_replica.cc:333] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.824894 13967 raft_consensus.cc:2243] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.825825 13967 raft_consensus.cc:2272] T 63e7d3ca52644473a44be3dc169dc371 P 210fa23e94494ef1b79aac9dc6bd3b9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.828006 13967 tablet_replica.cc:333] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f: stopping tablet replica
I20260709 13:01:58.828480 13967 raft_consensus.cc:2243] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.828999 13967 raft_consensus.cc:2272] T a0fe3f59937449dcb158c31de77239db P 210fa23e94494ef1b79aac9dc6bd3b9f [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:58.833316 14722 consensus_peers.cc:597] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 -> Peer 210fa23e94494ef1b79aac9dc6bd3b9f (127.13.163.194:33511): Couldn't send request to peer 210fa23e94494ef1b79aac9dc6bd3b9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:58.860546 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
I20260709 13:01:58.880168 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
I20260709 13:01:58.909526 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:58.910163 13967 tablet_replica.cc:333] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.910723 13967 raft_consensus.cc:2243] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.911229 13967 raft_consensus.cc:2272] T 150db11da6874fb78006d4ce045cdd23 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.913059 13967 tablet_replica.cc:333] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.913573 13967 raft_consensus.cc:2243] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.914000 13967 raft_consensus.cc:2272] T 991373c5f7b1425d9a025af65cf249e8 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.915603 13967 tablet_replica.cc:333] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.916112 13967 raft_consensus.cc:2243] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.917009 13967 raft_consensus.cc:2272] T e2f6bfa852bf48908597267c4efa2ee7 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.918751 13967 tablet_replica.cc:333] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.919269 13967 raft_consensus.cc:2243] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.919668 13967 raft_consensus.cc:2272] T 2ca06a8b56a0487594f730031fb98950 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.921336 13967 tablet_replica.cc:333] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.921846 13967 raft_consensus.cc:2243] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.922245 13967 raft_consensus.cc:2272] T 29a7233db7eb430695d20d08ffeba6b3 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.923797 13967 tablet_replica.cc:333] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.924259 13967 raft_consensus.cc:2243] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.925019 13967 raft_consensus.cc:2272] T 87fb42e4678a46d08559dd49eeaba53f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.926846 13967 tablet_replica.cc:333] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.927336 13967 raft_consensus.cc:2243] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:58.928006 13967 raft_consensus.cc:2272] T a0fe3f59937449dcb158c31de77239db P c9cecdd7c2ca4d95a4610ac2bce82947 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.929899 13967 tablet_replica.cc:333] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.930379 13967 raft_consensus.cc:2243] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.930815 13967 raft_consensus.cc:2272] T 63e7d3ca52644473a44be3dc169dc371 P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.933264 13967 tablet_replica.cc:333] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.933737 13967 raft_consensus.cc:2243] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.934093 13967 pending_rounds.cc:70] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Trying to abort 1 pending ops.
I20260709 13:01:58.934288 13967 pending_rounds.cc:77] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305634276380303360 op_type: NO_OP noop_request { }
I20260709 13:01:58.934613 13967 raft_consensus.cc:2889] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:01:58.934890 13967 raft_consensus.cc:2272] T ca11e7cb186347cf82d365cdcee6474f P c9cecdd7c2ca4d95a4610ac2bce82947 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.936932 13967 tablet_replica.cc:333] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947: stopping tablet replica
I20260709 13:01:58.937372 13967 raft_consensus.cc:2243] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.937788 13967 raft_consensus.cc:2272] T ad732f811171485182ce493d628ec16a P c9cecdd7c2ca4d95a4610ac2bce82947 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.966895 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:01:58.985855 13967 master.cc:562] Master@127.13.163.254:40561 shutting down...
I20260709 13:01:59.006453 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.007005 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.007369 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P f1a1dd5e133b415e837864b764847890: stopping tablet replica
I20260709 13:01:59.027966 13967 master.cc:584] Master@127.13.163.254:40561 shutdown complete.
210fa23e94494ef1b79aac9dc6bd3b9f  4
c9cecdd7c2ca4d95a4610ac2bce82947  1
caeaf499879d4d89a633aafc8d4f1263  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10829 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:01:59.069703 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:45133
I20260709 13:01:59.070931 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:59.076722 14961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.077049 14962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.077785 14964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:59.081219 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:59.086084 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:59.086364 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:59.086539 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602119086519 us; error 0 us; skew 500 ppm
I20260709 13:01:59.087180 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:59.092760 13967 webserver.cc:533] Webserver started at http://127.13.163.254:36217/ using document root <none> and password file <none>
I20260709 13:01:59.093472 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:59.093730 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:59.094112 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:59.095482 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "df53447e208f455db54704b341851d53"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.101435 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:01:59.104928 14969 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.105825 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:01:59.106132 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "df53447e208f455db54704b341851d53"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.106420 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:59.126739 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:59.127995 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:59.167663 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:45133
I20260709 13:01:59.167752 15020 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:45133 every 8 connection(s)
I20260709 13:01:59.171766 15021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.181937 15021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Bootstrap starting.
I20260709 13:01:59.186647 15021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:59.191227 15021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: No bootstrap required, opened a new log
I20260709 13:01:59.193446 15021 raft_consensus.cc:359] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER }
I20260709 13:01:59.193863 15021 raft_consensus.cc:385] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:59.194142 15021 raft_consensus.cc:740] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df53447e208f455db54704b341851d53, State: Initialized, Role: FOLLOWER
I20260709 13:01:59.194788 15021 consensus_queue.cc:260] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER }
I20260709 13:01:59.195228 15021 raft_consensus.cc:399] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:59.195523 15021 raft_consensus.cc:493] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:59.195796 15021 raft_consensus.cc:3060] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:59.200500 15021 raft_consensus.cc:515] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER }
I20260709 13:01:59.201190 15021 leader_election.cc:304] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df53447e208f455db54704b341851d53; no voters: 
I20260709 13:01:59.202349 15021 leader_election.cc:290] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:59.202842 15024 raft_consensus.cc:2804] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:59.204282 15024 raft_consensus.cc:697] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 1 LEADER]: Becoming Leader. State: Replica: df53447e208f455db54704b341851d53, State: Running, Role: LEADER
I20260709 13:01:59.205097 15024 consensus_queue.cc:237] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER }
I20260709 13:01:59.205473 15021 sys_catalog.cc:565] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:59.209796 15025 sys_catalog.cc:455] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df53447e208f455db54704b341851d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER } }
I20260709 13:01:59.209894 15026 sys_catalog.cc:455] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df53447e208f455db54704b341851d53. Latest consensus state: current_term: 1 leader_uuid: "df53447e208f455db54704b341851d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df53447e208f455db54704b341851d53" member_type: VOTER } }
I20260709 13:01:59.210669 15025 sys_catalog.cc:458] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:59.210899 15026 sys_catalog.cc:458] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:59.219759 15031 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:59.224686 15031 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:59.226414 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:59.234196 15031 catalog_manager.cc:1383] Generated new cluster ID: 8856f24f0f8a4279a812c94f94be4367
I20260709 13:01:59.234476 15031 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:59.249363 15031 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:59.250721 15031 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:59.266949 15031 catalog_manager.cc:6070] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Generated new TSK 0
I20260709 13:01:59.267690 15031 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:59.293184 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:59.299921 15042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.300972 15043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.302693 15045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:59.302911 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:59.304008 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:59.304244 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:59.304458 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602119304426 us; error 0 us; skew 500 ppm
I20260709 13:01:59.305078 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:59.307749 13967 webserver.cc:533] Webserver started at http://127.13.163.193:37453/ using document root <none> and password file <none>
I20260709 13:01:59.308368 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:59.308600 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:59.308943 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:59.310087 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "12daa67457954a0cab27e32bf85fa115"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.314962 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:01:59.319085 15050 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.320140 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:01:59.320446 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "12daa67457954a0cab27e32bf85fa115"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.320706 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:59.351975 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:59.353343 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:59.354910 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:59.357728 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:59.357942 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.358270 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:59.358448 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.405855 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:38845
I20260709 13:01:59.405984 15112 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:38845 every 8 connection(s)
I20260709 13:01:59.411219 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:59.418555 15117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.420915 15118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:59.422282 13967 server_base.cc:1061] running on GCE node
W20260709 13:01:59.423178 15120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:59.424300 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:59.424592 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:59.424801 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602119424781 us; error 0 us; skew 500 ppm
I20260709 13:01:59.425537 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:59.427351 15113 heartbeater.cc:344] Connected to a master server at 127.13.163.254:45133
I20260709 13:01:59.427760 15113 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:59.428637 13967 webserver.cc:533] Webserver started at http://127.13.163.194:42849/ using document root <none> and password file <none>
I20260709 13:01:59.428738 15113 heartbeater.cc:507] Master 127.13.163.254:45133 requested a full tablet report, sending...
I20260709 13:01:59.429384 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:59.429646 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:59.429994 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:59.431221 14986 ts_manager.cc:194] Registered new tserver with Master: 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:01:59.431315 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "3d45101e0851484690b4f03a4dcdb63d"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.433224 14986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59940
I20260709 13:01:59.437427 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:01:59.441465 15125 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.442371 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:01:59.442766 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "3d45101e0851484690b4f03a4dcdb63d"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.443063 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:59.463443 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:59.464751 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:59.466501 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:59.469312 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:59.469506 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.469713 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:59.469839 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.511492 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:38049
I20260709 13:01:59.511588 15187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:38049 every 8 connection(s)
I20260709 13:01:59.516294 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:59.523571 15191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.524811 15192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:59.527860 15194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:59.528963 13967 server_base.cc:1061] running on GCE node
I20260709 13:01:59.529027 15188 heartbeater.cc:344] Connected to a master server at 127.13.163.254:45133
I20260709 13:01:59.529645 15188 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:59.530153 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:59.530424 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:59.530481 15188 heartbeater.cc:507] Master 127.13.163.254:45133 requested a full tablet report, sending...
I20260709 13:01:59.530639 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602119530617 us; error 0 us; skew 500 ppm
I20260709 13:01:59.531447 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:59.533270 14986 ts_manager.cc:194] Registered new tserver with Master: 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:01:59.534649 13967 webserver.cc:533] Webserver started at http://127.13.163.195:35837/ using document root <none> and password file <none>
I20260709 13:01:59.535367 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:59.535336 14986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59952
I20260709 13:01:59.535830 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:59.536209 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:59.537704 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "4b633b84351f40de9437e3d532fe6fa7"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.542730 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:01:59.546526 15199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.547502 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:01:59.547951 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "4b633b84351f40de9437e3d532fe6fa7"
format_stamp: "Formatted at 2026-07-09 13:01:59 on dist-test-slave-w3b1"
I20260709 13:01:59.548238 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:59.561409 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:59.562757 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:59.564393 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:59.566995 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:59.567209 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.567554 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:59.567739 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:59.612610 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:42229
I20260709 13:01:59.612807 15261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:42229 every 8 connection(s)
I20260709 13:01:59.628023 15262 heartbeater.cc:344] Connected to a master server at 127.13.163.254:45133
I20260709 13:01:59.628429 15262 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:59.629302 15262 heartbeater.cc:507] Master 127.13.163.254:45133 requested a full tablet report, sending...
I20260709 13:01:59.631399 14986 ts_manager.cc:194] Registered new tserver with Master: 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:01:59.631830 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015802752s
I20260709 13:01:59.633395 14986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59958
I20260709 13:01:59.660909 14986 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59974:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:59.667922 14986 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:59.885996 15151 tablet_service.cc:1511] Processing CreateTablet for tablet af13e9e38f6a4785bef931855aafd50c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:59.887672 15149 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:59.889355 15147 tablet_service.cc:1511] Processing CreateTablet for tablet 06f7ae7c1d614611bb074a65fcc9a959 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:59.890159 15148 tablet_service.cc:1511] Processing CreateTablet for tablet bf60e792da464094a6aa2bfbcc5d71f2 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:59.892421 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 740a0a73cde54c9abf2460f0bbdcf02d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:59.894767 15150 tablet_service.cc:1511] Processing CreateTablet for tablet 15cd34bca3be499d87220d6e1b2a763f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:59.897140 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 5e777c555e1b4153913b5e6459164454 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:59.899524 15076 tablet_service.cc:1511] Processing CreateTablet for tablet af13e9e38f6a4785bef931855aafd50c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:59.899885 15078 tablet_service.cc:1511] Processing CreateTablet for tablet 5e777c555e1b4153913b5e6459164454 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:59.901791 15077 tablet_service.cc:1511] Processing CreateTablet for tablet 740a0a73cde54c9abf2460f0bbdcf02d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:59.902020 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 15cd34bca3be499d87220d6e1b2a763f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:59.903779 15072 tablet_service.cc:1511] Processing CreateTablet for tablet 06f7ae7c1d614611bb074a65fcc9a959 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:59.903944 15074 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:59.904726 15073 tablet_service.cc:1511] Processing CreateTablet for tablet bf60e792da464094a6aa2bfbcc5d71f2 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:59.907045 15071 tablet_service.cc:1511] Processing CreateTablet for tablet efef46e0b3c14af2a7179beba5c160b4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:59.908488 15145 tablet_service.cc:1511] Processing CreateTablet for tablet 211046e319804173a9b8954112b8d044 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:59.909148 15070 tablet_service.cc:1511] Processing CreateTablet for tablet 211046e319804173a9b8954112b8d044 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:59.909564 15144 tablet_service.cc:1511] Processing CreateTablet for tablet 661df855a98f493ca654519f6be2164f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:59.909710 15146 tablet_service.cc:1511] Processing CreateTablet for tablet efef46e0b3c14af2a7179beba5c160b4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:59.911115 15069 tablet_service.cc:1511] Processing CreateTablet for tablet 661df855a98f493ca654519f6be2164f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:59.912039 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13e9e38f6a4785bef931855aafd50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.912720 15072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7ae7c1d614611bb074a65fcc9a959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.913640 15070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211046e319804173a9b8954112b8d044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.914294 15078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e777c555e1b4153913b5e6459164454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.914918 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efef46e0b3c14af2a7179beba5c160b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.915557 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.921232 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 661df855a98f493ca654519f6be2164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.925197 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cd34bca3be499d87220d6e1b2a763f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.887674 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13e9e38f6a4785bef931855aafd50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.929179 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf60e792da464094a6aa2bfbcc5d71f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.929893 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740a0a73cde54c9abf2460f0bbdcf02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.930724 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 661df855a98f493ca654519f6be2164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.933200 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efef46e0b3c14af2a7179beba5c160b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.933893 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7ae7c1d614611bb074a65fcc9a959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.934783 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.937892 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740a0a73cde54c9abf2460f0bbdcf02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.945477 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211046e319804173a9b8954112b8d044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.949206 15148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf60e792da464094a6aa2bfbcc5d71f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.949930 15150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cd34bca3be499d87220d6e1b2a763f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.953195 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e777c555e1b4153913b5e6459164454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.984261 15227 tablet_service.cc:1511] Processing CreateTablet for tablet 5e777c555e1b4153913b5e6459164454 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:59.985718 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e777c555e1b4153913b5e6459164454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.990111 15226 tablet_service.cc:1511] Processing CreateTablet for tablet 740a0a73cde54c9abf2460f0bbdcf02d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:59.991526 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740a0a73cde54c9abf2460f0bbdcf02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:59.994874 15221 tablet_service.cc:1511] Processing CreateTablet for tablet 06f7ae7c1d614611bb074a65fcc9a959 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:59.996233 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7ae7c1d614611bb074a65fcc9a959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.000403 15153 tablet_service.cc:1511] Processing CreateTablet for tablet d4010b839395412ca61c77e701fcb965 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:02:00.002257 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4010b839395412ca61c77e701fcb965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.001952 15222 tablet_service.cc:1511] Processing CreateTablet for tablet bf60e792da464094a6aa2bfbcc5d71f2 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:02:00.003399 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf60e792da464094a6aa2bfbcc5d71f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.006084 15223 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:02:00.007566 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.008810 15075 tablet_service.cc:1511] Processing CreateTablet for tablet d4010b839395412ca61c77e701fcb965 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:02:00.010093 15224 tablet_service.cc:1511] Processing CreateTablet for tablet 15cd34bca3be499d87220d6e1b2a763f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:02:00.010645 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4010b839395412ca61c77e701fcb965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.011590 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cd34bca3be499d87220d6e1b2a763f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.018914 15220 tablet_service.cc:1511] Processing CreateTablet for tablet efef46e0b3c14af2a7179beba5c160b4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:02:00.020423 15220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efef46e0b3c14af2a7179beba5c160b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.014062 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 211046e319804173a9b8954112b8d044 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:02:00.023072 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211046e319804173a9b8954112b8d044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.025509 15076 tablet_service.cc:1511] Processing CreateTablet for tablet 447fc877e0f944fbb59ddd85ad25d8a9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:02:00.027055 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447fc877e0f944fbb59ddd85ad25d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.025730 15146 tablet_service.cc:1511] Processing CreateTablet for tablet 447fc877e0f944fbb59ddd85ad25d8a9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:02:00.027947 15225 tablet_service.cc:1511] Processing CreateTablet for tablet af13e9e38f6a4785bef931855aafd50c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:02:00.031731 15225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13e9e38f6a4785bef931855aafd50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.036695 15144 tablet_service.cc:1511] Processing CreateTablet for tablet 1868a2755dea467c9c719191d9927234 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:02:00.038718 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1868a2755dea467c9c719191d9927234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.060920 15218 tablet_service.cc:1511] Processing CreateTablet for tablet 661df855a98f493ca654519f6be2164f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:02:00.062295 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 661df855a98f493ca654519f6be2164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.034446 15072 tablet_service.cc:1511] Processing CreateTablet for tablet 16dab822009a4f00a0a480a48136ae25 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:02:00.065954 15072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16dab822009a4f00a0a480a48136ae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.069644 15149 tablet_service.cc:1511] Processing CreateTablet for tablet 16dab822009a4f00a0a480a48136ae25 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:02:00.073648 15278 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:00.074149 15071 tablet_service.cc:1511] Processing CreateTablet for tablet 09eea6ae09f04b0b9937f09218fcb3b1 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:02:00.075766 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09eea6ae09f04b0b9937f09218fcb3b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.089318 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447fc877e0f944fbb59ddd85ad25d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.094802 15151 tablet_service.cc:1511] Processing CreateTablet for tablet 09eea6ae09f04b0b9937f09218fcb3b1 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:02:00.096453 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09eea6ae09f04b0b9937f09218fcb3b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.112149 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16dab822009a4f00a0a480a48136ae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.112965 15073 tablet_service.cc:1511] Processing CreateTablet for tablet 73ae83bb85154bb08a2889f0f9f99cd8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:02:00.117537 15069 tablet_service.cc:1511] Processing CreateTablet for tablet 1868a2755dea467c9c719191d9927234 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:02:00.113868 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 73ae83bb85154bb08a2889f0f9f99cd8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:02:00.119199 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ae83bb85154bb08a2889f0f9f99cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.120155 15071 tablet_service.cc:1511] Processing CreateTablet for tablet c84610e2d65945cab0886909d43d5e9e (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:02:00.118116 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ae83bb85154bb08a2889f0f9f99cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.121512 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84610e2d65945cab0886909d43d5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.123957 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 365e6a51b3034d0483fdd8fc4cd362ea (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:02:00.133756 15074 tablet_service.cc:1511] Processing CreateTablet for tablet 365e6a51b3034d0483fdd8fc4cd362ea (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:02:00.140043 15278 tablet_bootstrap.cc:654] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.146908 15070 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8fe0d92f6f47dfb304a24bf915e2db (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:02:00.162484 15222 tablet_service.cc:1511] Processing CreateTablet for tablet d4010b839395412ca61c77e701fcb965 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:02:00.164938 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1868a2755dea467c9c719191d9927234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.166116 15070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8fe0d92f6f47dfb304a24bf915e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.170801 15078 tablet_service.cc:1511] Processing CreateTablet for tablet 4d62d65616394d059c479b07e82eccb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:02:00.182843 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4010b839395412ca61c77e701fcb965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.194212 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365e6a51b3034d0483fdd8fc4cd362ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.198061 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365e6a51b3034d0483fdd8fc4cd362ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.198243 15279 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:00.200980 15277 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:00.201107 15145 tablet_service.cc:1511] Processing CreateTablet for tablet c84610e2d65945cab0886909d43d5e9e (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:02:00.202643 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84610e2d65945cab0886909d43d5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.205740 15224 tablet_service.cc:1511] Processing CreateTablet for tablet 447fc877e0f944fbb59ddd85ad25d8a9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:02:00.207365 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447fc877e0f944fbb59ddd85ad25d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.207458 15147 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8fe0d92f6f47dfb304a24bf915e2db (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:02:00.219347 15277 tablet_bootstrap.cc:654] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.220881 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8fe0d92f6f47dfb304a24bf915e2db. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:02:00.221565 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:00.222072 15223 tablet_service.cc:1511] Processing CreateTablet for tablet 73ae83bb85154bb08a2889f0f9f99cd8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:02:00.223634 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ae83bb85154bb08a2889f0f9f99cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.226156 15151 tablet_service.cc:1511] Processing CreateTablet for tablet 4d62d65616394d059c479b07e82eccb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:02:00.227660 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d62d65616394d059c479b07e82eccb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.234699 15077 tablet_service.cc:1511] Processing CreateTablet for tablet eaa49b1ae77944d7b6866e819d63d466 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:02:00.236217 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa49b1ae77944d7b6866e819d63d466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.216624 15279 tablet_bootstrap.cc:654] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.228276 15148 tablet_service.cc:1511] Processing CreateTablet for tablet eaa49b1ae77944d7b6866e819d63d466 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:02:00.249547 15227 tablet_service.cc:1511] Processing CreateTablet for tablet 09eea6ae09f04b0b9937f09218fcb3b1 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:02:00.251001 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09eea6ae09f04b0b9937f09218fcb3b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.251549 15076 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0c065a7fec4f5c88b4bc85b3508231 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:02:00.253006 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0c065a7fec4f5c88b4bc85b3508231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.259356 15078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d62d65616394d059c479b07e82eccb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.248976 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 41c6ed4c81c041d18f143b6cd51bea35 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:02:00.261969 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c6ed4c81c041d18f143b6cd51bea35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.264940 15148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa49b1ae77944d7b6866e819d63d466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.271112 15150 tablet_service.cc:1511] Processing CreateTablet for tablet 41c6ed4c81c041d18f143b6cd51bea35 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:02:00.272557 15150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c6ed4c81c041d18f143b6cd51bea35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.275050 15144 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0c065a7fec4f5c88b4bc85b3508231 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:02:00.288110 15221 tablet_service.cc:1511] Processing CreateTablet for tablet 365e6a51b3034d0483fdd8fc4cd362ea (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:02:00.291666 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365e6a51b3034d0483fdd8fc4cd362ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.293387 15218 tablet_service.cc:1511] Processing CreateTablet for tablet 4d62d65616394d059c479b07e82eccb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:02:00.296089 15073 tablet_service.cc:1511] Processing CreateTablet for tablet 9a20274fd45144db87032cdf948f324d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:02:00.305750 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a20274fd45144db87032cdf948f324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.305413 15222 tablet_service.cc:1511] Processing CreateTablet for tablet 1868a2755dea467c9c719191d9927234 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:02:00.297940 15225 tablet_service.cc:1511] Processing CreateTablet for tablet c84610e2d65945cab0886909d43d5e9e (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:02:00.260692 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 16dab822009a4f00a0a480a48136ae25 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:02:00.289388 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0c065a7fec4f5c88b4bc85b3508231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.308854 15220 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8fe0d92f6f47dfb304a24bf915e2db (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:02:00.341706 15224 tablet_service.cc:1511] Processing CreateTablet for tablet eaa49b1ae77944d7b6866e819d63d466 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:02:00.336566 15226 tablet_service.cc:1511] Processing CreateTablet for tablet 41c6ed4c81c041d18f143b6cd51bea35 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:02:00.354048 15071 tablet_service.cc:1511] Processing CreateTablet for tablet 301fa97ba426402092eaf85685303b33 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:02:00.359786 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1868a2755dea467c9c719191d9927234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.360626 15146 tablet_service.cc:1511] Processing CreateTablet for tablet f52fa48697b145c98311ef7704e35002 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:02:00.362134 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52fa48697b145c98311ef7704e35002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.363554 15277 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:00.364138 15277 ts_tablet_manager.cc:1403] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.190s	user 0.006s	sys 0.008s
I20260709 13:02:00.365211 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d62d65616394d059c479b07e82eccb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.367713 15277 raft_consensus.cc:359] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.368645 15277 raft_consensus.cc:385] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:00.369154 15277 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:00.370265 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16dab822009a4f00a0a480a48136ae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.373762 15277 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.377140 15225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84610e2d65945cab0886909d43d5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.378680 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:02:00.380182 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.387590 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301fa97ba426402092eaf85685303b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.391973 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 808dd81f1b604329848aa94e3ecedc36 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:02:00.400820 15222 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0c065a7fec4f5c88b4bc85b3508231 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:02:00.401289 15278 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:00.401890 15278 ts_tablet_manager.cc:1403] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.329s	user 0.015s	sys 0.002s
I20260709 13:02:00.401701 15078 tablet_service.cc:1511] Processing CreateTablet for tablet f52fa48697b145c98311ef7704e35002 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:02:00.404805 15278 raft_consensus.cc:359] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.405632 15278 raft_consensus.cc:385] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:00.405936 15278 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:00.406555 15149 tablet_service.cc:1511] Processing CreateTablet for tablet 9a20274fd45144db87032cdf948f324d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:02:00.397570 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c6ed4c81c041d18f143b6cd51bea35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.407928 15278 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.415911 15147 tablet_service.cc:1511] Processing CreateTablet for tablet 301fa97ba426402092eaf85685303b33 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:02:00.422822 15279 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:00.425935 15279 ts_tablet_manager.cc:1403] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.228s	user 0.021s	sys 0.028s
I20260709 13:02:00.426127 15078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52fa48697b145c98311ef7704e35002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.427780 15077 tablet_service.cc:1511] Processing CreateTablet for tablet af12fc69de0b47b9adaf0267fb143776 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:02:00.429661 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:02:00.429862 15227 tablet_service.cc:1511] Processing CreateTablet for tablet 9a20274fd45144db87032cdf948f324d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:02:00.430588 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a20274fd45144db87032cdf948f324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.431030 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.432520 15074 tablet_service.cc:1511] Processing CreateTablet for tablet ec09a7a4d59c4296aa02dde6bd37f81c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:02:00.434100 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec09a7a4d59c4296aa02dde6bd37f81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.436314 15113 heartbeater.cc:499] Master 127.13.163.254:45133 was elected leader, sending a full tablet report...
I20260709 13:02:00.437206 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808dd81f1b604329848aa94e3ecedc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.437997 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af12fc69de0b47b9adaf0267fb143776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.431392 15070 tablet_service.cc:1511] Processing CreateTablet for tablet ee2d00b915f04988a8794ab25f22cbd5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:02:00.449432 15070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2d00b915f04988a8794ab25f22cbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.448956 15279 raft_consensus.cc:359] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.422384 15072 tablet_service.cc:1511] Processing CreateTablet for tablet 808dd81f1b604329848aa94e3ecedc36 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:02:00.453017 15279 raft_consensus.cc:385] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:00.453423 15279 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:00.454285 15279 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.455971 15072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808dd81f1b604329848aa94e3ecedc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.451792 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0c065a7fec4f5c88b4bc85b3508231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.463601 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa49b1ae77944d7b6866e819d63d466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.452090 15069 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6a82b88a6f4f6093256a0d85708d37 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:02:00.465649 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6a82b88a6f4f6093256a0d85708d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.422544 15145 tablet_service.cc:1511] Processing CreateTablet for tablet af12fc69de0b47b9adaf0267fb143776 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:02:00.469215 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a20274fd45144db87032cdf948f324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.469830 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af12fc69de0b47b9adaf0267fb143776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.473174 15220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8fe0d92f6f47dfb304a24bf915e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.477499 15144 tablet_service.cc:1511] Processing CreateTablet for tablet ee2d00b915f04988a8794ab25f22cbd5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:02:00.490387 15076 tablet_service.cc:1511] Processing CreateTablet for tablet d50a21b24380415781c0e63c5d613aff (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:02:00.488030 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301fa97ba426402092eaf85685303b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.496927 15278 ts_tablet_manager.cc:1434] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.095s	user 0.010s	sys 0.005s
I20260709 13:02:00.507583 15277 ts_tablet_manager.cc:1434] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.143s	user 0.016s	sys 0.003s
I20260709 13:02:00.507989 15188 heartbeater.cc:499] Master 127.13.163.254:45133 was elected leader, sending a full tablet report...
I20260709 13:02:00.508916 15277 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:00.507640 15278 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:00.511487 15148 tablet_service.cc:1511] Processing CreateTablet for tablet 14bd17b479354abca0014ae059714e8f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:02:00.508955 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50a21b24380415781c0e63c5d613aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.501771 15073 tablet_service.cc:1511] Processing CreateTablet for tablet 14bd17b479354abca0014ae059714e8f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:02:00.533051 15262 heartbeater.cc:499] Master 127.13.163.254:45133 was elected leader, sending a full tablet report...
I20260709 13:02:00.537243 15283 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.537817 15283 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.542932 15146 tablet_service.cc:1511] Processing CreateTablet for tablet ec09a7a4d59c4296aa02dde6bd37f81c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:02:00.540695 15283 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:00.549238 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bd17b479354abca0014ae059714e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.556922 15148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bd17b479354abca0014ae059714e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.557888 15151 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6a82b88a6f4f6093256a0d85708d37 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:02:00.561815 15223 tablet_service.cc:1511] Processing CreateTablet for tablet f52fa48697b145c98311ef7704e35002 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:02:00.563254 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52fa48697b145c98311ef7704e35002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.600351 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:02:00.601905 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.589888 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec09a7a4d59c4296aa02dde6bd37f81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.600333 15277 tablet_bootstrap.cc:654] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.618237 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2d00b915f04988a8794ab25f22cbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.629914 15218 tablet_service.cc:1511] Processing CreateTablet for tablet 808dd81f1b604329848aa94e3ecedc36 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:02:00.631349 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7fd74046f492f8ef8366fbc675ee0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:02:00.632681 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7fd74046f492f8ef8366fbc675ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.635993 15224 tablet_service.cc:1511] Processing CreateTablet for tablet 301fa97ba426402092eaf85685303b33 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:02:00.641184 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6a82b88a6f4f6093256a0d85708d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.651389 15279 ts_tablet_manager.cc:1434] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.225s	user 0.017s	sys 0.011s
I20260709 13:02:00.665630 15285 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.666143 15285 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.631392 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808dd81f1b604329848aa94e3ecedc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.670866 15285 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:00.707629 15145 tablet_service.cc:1511] Processing CreateTablet for tablet 08bf5d5f44ec498499e1cb1be55c2283 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:02:00.708389 15278 tablet_bootstrap.cc:654] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.708812 15078 tablet_service.cc:1511] Processing CreateTablet for tablet 08bf5d5f44ec498499e1cb1be55c2283 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:02:00.709311 15279 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:00.712496 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bf5d5f44ec498499e1cb1be55c2283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.712105 15221 tablet_service.cc:1511] Processing CreateTablet for tablet af12fc69de0b47b9adaf0267fb143776 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:02:00.712936 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:00.670758 15071 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7fd74046f492f8ef8366fbc675ee0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 13:02:00.728986 15051 leader_election.cc:343] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:00.714506 15077 tablet_service.cc:1511] Processing CreateTablet for tablet 30c8fd833bca41f69ed253599a8889b9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:02:00.712055 15225 tablet_service.cc:1511] Processing CreateTablet for tablet ee2d00b915f04988a8794ab25f22cbd5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:02:00.724295 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:00.731637 15150 tablet_service.cc:1511] Processing CreateTablet for tablet d50a21b24380415781c0e63c5d613aff (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:02:00.731671 15054 leader_election.cc:343] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:00.733085 15150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50a21b24380415781c0e63c5d613aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.733147 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:00.734437 15283 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:00.725072 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301fa97ba426402092eaf85685303b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.757308 15222 tablet_service.cc:1511] Processing CreateTablet for tablet 14bd17b479354abca0014ae059714e8f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:02:00.746361 15227 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6a82b88a6f4f6093256a0d85708d37 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:02:00.733462 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 30c8fd833bca41f69ed253599a8889b9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:02:00.744103 15220 tablet_service.cc:1511] Processing CreateTablet for tablet ec09a7a4d59c4296aa02dde6bd37f81c (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:02:00.762212 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af12fc69de0b47b9adaf0267fb143776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.768456 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30c8fd833bca41f69ed253599a8889b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.768909 15226 tablet_service.cc:1511] Processing CreateTablet for tablet d50a21b24380415781c0e63c5d613aff (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:02:00.770839 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7fd74046f492f8ef8366fbc675ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.775832 15279 tablet_bootstrap.cc:654] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.777482 15078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bf5d5f44ec498499e1cb1be55c2283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.778684 15069 tablet_service.cc:1511] Processing CreateTablet for tablet de505aa1e895447486f53629c5269998 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:02:00.789857 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de505aa1e895447486f53629c5269998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.781576 15225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2d00b915f04988a8794ab25f22cbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.782469 15074 tablet_service.cc:1511] Processing CreateTablet for tablet cf9063404f3d4db9b81ce66060eee5e7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:02:00.792902 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6a82b88a6f4f6093256a0d85708d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.793308 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9063404f3d4db9b81ce66060eee5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.794646 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50a21b24380415781c0e63c5d613aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.796586 15153 tablet_service.cc:1511] Processing CreateTablet for tablet de505aa1e895447486f53629c5269998 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:02:00.797173 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30c8fd833bca41f69ed253599a8889b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.797963 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de505aa1e895447486f53629c5269998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.802673 15147 tablet_service.cc:1511] Processing CreateTablet for tablet cf9063404f3d4db9b81ce66060eee5e7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:02:00.804140 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9063404f3d4db9b81ce66060eee5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.806452 15144 tablet_service.cc:1511] Processing CreateTablet for tablet 4150ed0096c74899b55a5128304e91f6 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:02:00.811048 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bd17b479354abca0014ae059714e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.813198 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150ed0096c74899b55a5128304e91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.814875 15148 tablet_service.cc:1511] Processing CreateTablet for tablet b358d067cea0428190a484601dea7c69 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:02:00.826483 15277 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:00.833456 15220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec09a7a4d59c4296aa02dde6bd37f81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.833273 15151 tablet_service.cc:1511] Processing CreateTablet for tablet 0afe5ac407a04216b625837a5bca525f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:02:00.819430 15149 tablet_service.cc:1511] Processing CreateTablet for tablet a12ada8057b141518113efeb519c6514 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:02:00.846943 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:00.851642 15076 tablet_service.cc:1511] Processing CreateTablet for tablet b358d067cea0428190a484601dea7c69 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:02:00.854795 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 4150ed0096c74899b55a5128304e91f6 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:02:00.860466 15073 tablet_service.cc:1511] Processing CreateTablet for tablet a12ada8057b141518113efeb519c6514 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:02:00.863631 15284 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.864197 15284 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.865274 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:00.866952 15284 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:00.867209 15202 leader_election.cc:343] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:00.866008 15072 tablet_service.cc:1511] Processing CreateTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 13:02:00.881515 15203 leader_election.cc:343] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:00.882421 15223 tablet_service.cc:1511] Processing CreateTablet for tablet 08bf5d5f44ec498499e1cb1be55c2283 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:02:00.883838 15203 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:00.884618 15070 tablet_service.cc:1511] Processing CreateTablet for tablet 0afe5ac407a04216b625837a5bca525f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:02:00.870887 15277 ts_tablet_manager.cc:1403] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.361s	user 0.017s	sys 0.010s
I20260709 13:02:00.888371 15285 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:00.891331 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12ada8057b141518113efeb519c6514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.893790 15148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b358d067cea0428190a484601dea7c69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.885632 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b358d067cea0428190a484601dea7c69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.883834 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bf5d5f44ec498499e1cb1be55c2283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.896333 15277 raft_consensus.cc:359] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.907694 15277 raft_consensus.cc:385] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:00.908082 15277 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:00.908252 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12ada8057b141518113efeb519c6514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.909111 15277 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.910400 15227 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7fd74046f492f8ef8366fbc675ee0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:02:00.911561 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150ed0096c74899b55a5128304e91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.929350 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7fd74046f492f8ef8366fbc675ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.926159 15277 ts_tablet_manager.cc:1434] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:02:00.935580 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 30c8fd833bca41f69ed253599a8889b9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:02:00.924881 15070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afe5ac407a04216b625837a5bca525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.941197 15072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3342a5b2ad49e2b252d7cd47b9851d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.949908 15278 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:00.952729 15278 ts_tablet_manager.cc:1403] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.445s	user 0.017s	sys 0.007s
I20260709 13:02:00.957115 15145 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:02:00.958220 15278 raft_consensus.cc:359] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.959087 15278 raft_consensus.cc:385] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:00.959923 15278 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:00.960777 15278 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:00.963920 15277 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:00.967478 15069 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:02:00.954238 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30c8fd833bca41f69ed253599a8889b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.948737 15150 tablet_service.cc:1511] Processing CreateTablet for tablet 42a84a565612402dbb559ec973692018 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:02:00.996620 15279 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:00.997213 15279 ts_tablet_manager.cc:1403] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.288s	user 0.020s	sys 0.005s
I20260709 13:02:00.998070 15277 tablet_bootstrap.cc:654] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:00.997916 15147 tablet_service.cc:1511] Processing CreateTablet for tablet 30853833ff254a12bcc77a177a964bb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:02:00.999142 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afe5ac407a04216b625837a5bca525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.999718 15077 tablet_service.cc:1511] Processing CreateTablet for tablet 07a1de4f2ab24624aa87e5619961710d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:02:01.000237 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.981618 15226 tablet_service.cc:1511] Processing CreateTablet for tablet de505aa1e895447486f53629c5269998 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:02:00.983314 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 60df009dd21a4cb3867d92cb3a69c1e4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:02:00.984273 15146 tablet_service.cc:1511] Processing CreateTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:02:01.002034 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de505aa1e895447486f53629c5269998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.004122 15222 tablet_service.cc:1511] Processing CreateTablet for tablet 4150ed0096c74899b55a5128304e91f6 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:02:01.005558 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150ed0096c74899b55a5128304e91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:00.987401 15218 tablet_service.cc:1511] Processing CreateTablet for tablet cf9063404f3d4db9b81ce66060eee5e7 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:02:01.005707 15071 tablet_service.cc:1511] Processing CreateTablet for tablet 42a84a565612402dbb559ec973692018 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:02:01.007820 15073 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5f358b91ee427880ebecaca8118e4f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:02:01.018524 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:01.020186 15128 leader_election.cc:343] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.008050 15227 tablet_service.cc:1511] Processing CreateTablet for tablet b358d067cea0428190a484601dea7c69 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:02:01.000147 15279 raft_consensus.cc:359] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.021746 15279 raft_consensus.cc:385] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.022274 15279 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.023330 15279 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.024118 15221 tablet_service.cc:1511] Processing CreateTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:02:01.026212 15278 ts_tablet_manager.cc:1434] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.000s
I20260709 13:02:00.988404 15220 tablet_service.cc:1511] Processing CreateTablet for tablet 0afe5ac407a04216b625837a5bca525f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:02:01.028718 15279 ts_tablet_manager.cc:1434] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 13:02:01.029632 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.029867 15279 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:00.984546 15070 tablet_service.cc:1511] Processing CreateTablet for tablet 30853833ff254a12bcc77a177a964bb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:02:00.985284 15074 tablet_service.cc:1511] Processing CreateTablet for tablet 60df009dd21a4cb3867d92cb3a69c1e4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:02:01.029906 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.013960 15224 tablet_service.cc:1511] Processing CreateTablet for tablet a12ada8057b141518113efeb519c6514 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:02:01.037114 15150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a84a565612402dbb559ec973692018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.037644 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12ada8057b141518113efeb519c6514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.043524 15144 tablet_service.cc:1511] Processing CreateTablet for tablet 07a1de4f2ab24624aa87e5619961710d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:02:01.044924 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9063404f3d4db9b81ce66060eee5e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:02:01.046700 15126 leader_election.cc:343] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.047158 15126 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:01.047956 15284 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:01.049116 15078 tablet_service.cc:1511] Processing CreateTablet for tablet 7defae86b18344bdb8b04e0d7c8f1a8d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:02:01.050467 15220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afe5ac407a04216b625837a5bca525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.043350 15279 tablet_bootstrap.cc:654] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.051657 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b358d067cea0428190a484601dea7c69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.057489 15278 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.060425 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a1de4f2ab24624aa87e5619961710d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.061094 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df009dd21a4cb3867d92cb3a69c1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.065235 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5f358b91ee427880ebecaca8118e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.068965 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a1de4f2ab24624aa87e5619961710d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.069166 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a84a565612402dbb559ec973692018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.073166 15070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30853833ff254a12bcc77a177a964bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.073428 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3342a5b2ad49e2b252d7cd47b9851d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.074323 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5f358b91ee427880ebecaca8118e4f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:02:01.075613 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5f358b91ee427880ebecaca8118e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.077459 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30853833ff254a12bcc77a177a964bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.089184 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df009dd21a4cb3867d92cb3a69c1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.092440 15072 tablet_service.cc:1511] Processing CreateTablet for tablet f11eb27eddd0475ca037593dcdb66376 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:02:01.093902 15072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11eb27eddd0475ca037593dcdb66376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.094350 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3342a5b2ad49e2b252d7cd47b9851d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.102046 15223 tablet_service.cc:1511] Processing CreateTablet for tablet 42a84a565612402dbb559ec973692018 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:02:01.106899 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a84a565612402dbb559ec973692018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.111244 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 07a1de4f2ab24624aa87e5619961710d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:02:01.123544 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a1de4f2ab24624aa87e5619961710d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.107158 15277 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.125273 15277 ts_tablet_manager.cc:1403] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.162s	user 0.007s	sys 0.008s
I20260709 13:02:01.113166 15078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7defae86b18344bdb8b04e0d7c8f1a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.115576 15149 tablet_service.cc:1511] Processing CreateTablet for tablet 7defae86b18344bdb8b04e0d7c8f1a8d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:02:01.134578 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 639c51e1c4da436b99c010fb41cebc63 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:02:01.135903 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 639c51e1c4da436b99c010fb41cebc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.137634 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7defae86b18344bdb8b04e0d7c8f1a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.106662 15225 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:02:01.117750 15283 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.138943 15283 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.141186 15283 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.142848 15279 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.143357 15279 ts_tablet_manager.cc:1403] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.114s	user 0.009s	sys 0.002s
I20260709 13:02:01.144430 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.145646 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:01.147258 15051 leader_election.cc:343] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:01.148411 15054 leader_election.cc:343] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.115818 15076 tablet_service.cc:1511] Processing CreateTablet for tablet ca6bbc5df3084af4a938113f16eeedb5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:02:01.148864 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:01.149610 15076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6bbc5df3084af4a938113f16eeedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.150079 15283 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:01.152990 15278 tablet_bootstrap.cc:654] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.139400 15225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.128391 15277 raft_consensus.cc:359] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.160984 15277 raft_consensus.cc:385] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.161381 15277 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.171697 15277 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.174808 15150 tablet_service.cc:1511] Processing CreateTablet for tablet 639c51e1c4da436b99c010fb41cebc63 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:02:01.176091 15150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 639c51e1c4da436b99c010fb41cebc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.180598 15069 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4be666e4a740bd853d8f2748ad6191 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:02:01.162355 15279 raft_consensus.cc:359] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.189679 15279 raft_consensus.cc:385] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.190152 15279 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.166492 15145 tablet_service.cc:1511] Processing CreateTablet for tablet ca6bbc5df3084af4a938113f16eeedb5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:02:01.168289 15077 tablet_service.cc:1511] Processing CreateTablet for tablet fe0a67b27bdc4c72abddb618266992a8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:02:01.192143 15145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6bbc5df3084af4a938113f16eeedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.192382 15077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0a67b27bdc4c72abddb618266992a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.155943 15148 tablet_service.cc:1511] Processing CreateTablet for tablet f11eb27eddd0475ca037593dcdb66376 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:02:01.161964 15218 tablet_service.cc:1511] Processing CreateTablet for tablet 60df009dd21a4cb3867d92cb3a69c1e4 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:02:01.194698 15148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11eb27eddd0475ca037593dcdb66376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.194808 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df009dd21a4cb3867d92cb3a69c1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.198181 15283 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.198935 15283 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.200726 15277 ts_tablet_manager.cc:1434] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.075s	user 0.003s	sys 0.005s
I20260709 13:02:01.201320 15069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4be666e4a740bd853d8f2748ad6191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.202911 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.187227 15226 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5f358b91ee427880ebecaca8118e4f (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:02:01.204746 15054 leader_election.cc:343] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.205333 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5f358b91ee427880ebecaca8118e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.205813 15151 tablet_service.cc:1511] Processing CreateTablet for tablet fe0a67b27bdc4c72abddb618266992a8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:02:01.207548 15151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0a67b27bdc4c72abddb618266992a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.209007 15283 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.209556 15279 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.210345 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.253230 15279 ts_tablet_manager.cc:1434] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.110s	user 0.009s	sys 0.001s
W20260709 13:02:01.255178 15051 leader_election.cc:343] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.255669 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:01.256026 15152 tablet_service.cc:1511] Processing CreateTablet for tablet 7f14609939d04f1e90837cb17fe19457 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:02:01.257443 15152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f14609939d04f1e90837cb17fe19457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.257397 15283 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:01.214970 15278 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.216831 15227 tablet_service.cc:1511] Processing CreateTablet for tablet f11eb27eddd0475ca037593dcdb66376 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:02:01.217788 15074 tablet_service.cc:1511] Processing CreateTablet for tablet eb265c17a8bf4acfb7e480e4cf91d39b (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:02:01.222281 15153 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4be666e4a740bd853d8f2748ad6191 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:02:01.259677 15278 ts_tablet_manager.cc:1403] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.202s	user 0.009s	sys 0.005s
I20260709 13:02:01.260388 15153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4be666e4a740bd853d8f2748ad6191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.262874 15278 raft_consensus.cc:359] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:01.224246 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:01.225770 15221 tablet_service.cc:1511] Processing CreateTablet for tablet 7defae86b18344bdb8b04e0d7c8f1a8d (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:02:01.265383 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb265c17a8bf4acfb7e480e4cf91d39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.227523 15073 tablet_service.cc:1511] Processing CreateTablet for tablet 7f14609939d04f1e90837cb17fe19457 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:02:01.266522 15073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f14609939d04f1e90837cb17fe19457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.229522 15071 tablet_service.cc:1511] Processing CreateTablet for tablet d386fde0d10e4abf8b63290770e3ede0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:02:01.238796 15222 tablet_service.cc:1511] Processing CreateTablet for tablet 30853833ff254a12bcc77a177a964bb9 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:02:01.222543 15146 tablet_service.cc:1511] Processing CreateTablet for tablet eb265c17a8bf4acfb7e480e4cf91d39b (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:02:01.247686 15224 tablet_service.cc:1511] Processing CreateTablet for tablet ca6bbc5df3084af4a938113f16eeedb5 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:02:01.273871 15071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d386fde0d10e4abf8b63290770e3ede0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.264330 15227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11eb27eddd0475ca037593dcdb66376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.277228 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7defae86b18344bdb8b04e0d7c8f1a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.276994 15220 tablet_service.cc:1511] Processing CreateTablet for tablet 639c51e1c4da436b99c010fb41cebc63 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:02:01.283062 15220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 639c51e1c4da436b99c010fb41cebc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.264441 15279 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.297317 15222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30853833ff254a12bcc77a177a964bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.300817 15224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6bbc5df3084af4a938113f16eeedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.274242 15146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb265c17a8bf4acfb7e480e4cf91d39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.304975 15278 raft_consensus.cc:385] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.305411 15278 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.306022 15147 tablet_service.cc:1511] Processing CreateTablet for tablet d386fde0d10e4abf8b63290770e3ede0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:02:01.308925 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d386fde0d10e4abf8b63290770e3ede0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.245810 15277 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.306389 15278 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.342815 15278 ts_tablet_manager.cc:1434] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.083s	user 0.004s	sys 0.002s
I20260709 13:02:01.343881 15278 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.354238 15279 tablet_bootstrap.cc:654] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.350627 15226 tablet_service.cc:1511] Processing CreateTablet for tablet fe0a67b27bdc4c72abddb618266992a8 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:02:01.358210 15226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0a67b27bdc4c72abddb618266992a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.363391 15277 tablet_bootstrap.cc:654] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.364969 15223 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4be666e4a740bd853d8f2748ad6191 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:02:01.366724 15278 tablet_bootstrap.cc:654] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.378275 15221 tablet_service.cc:1511] Processing CreateTablet for tablet eb265c17a8bf4acfb7e480e4cf91d39b (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:02:01.387663 15221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb265c17a8bf4acfb7e480e4cf91d39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.389067 15219 tablet_service.cc:1511] Processing CreateTablet for tablet 7f14609939d04f1e90837cb17fe19457 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:02:01.391314 15218 tablet_service.cc:1511] Processing CreateTablet for tablet d386fde0d10e4abf8b63290770e3ede0 (DEFAULT_TABLE table=test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:02:01.392941 15223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4be666e4a740bd853d8f2748ad6191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.393549 15284 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.395834 15284 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.398262 15284 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.399056 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.399412 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.399994 15237 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.401297 15126 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:02:01.401979 15128 leader_election.cc:343] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.403232 15284 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.403631 15284 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.403980 15284 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.406069 15219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f14609939d04f1e90837cb17fe19457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.406896 15218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d386fde0d10e4abf8b63290770e3ede0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.424701 15285 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.425704 15278 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.425361 15285 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.426234 15278 ts_tablet_manager.cc:1403] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.010s
I20260709 13:02:01.427666 15285 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.428432 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.429073 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:01.430131 15202 leader_election.cc:343] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:01.432860 15203 leader_election.cc:343] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.434595 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:01.435140 15277 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.435307 15279 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.435650 15277 ts_tablet_manager.cc:1403] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.190s	user 0.016s	sys 0.000s
I20260709 13:02:01.435788 15279 ts_tablet_manager.cc:1403] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.172s	user 0.012s	sys 0.001s
I20260709 13:02:01.435791 15285 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:01.439086 15279 raft_consensus.cc:359] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.439873 15279 raft_consensus.cc:385] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.440207 15279 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.441058 15279 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.443437 15279 ts_tablet_manager.cc:1434] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:02:01.444631 15277 raft_consensus.cc:359] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.445534 15277 raft_consensus.cc:385] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.445853 15277 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.446661 15277 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.448829 15284 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.429869 15278 raft_consensus.cc:359] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.459847 15278 raft_consensus.cc:385] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.460202 15278 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.460778 15277 ts_tablet_manager.cc:1434] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:02:01.461412 15284 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.461097 15278 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.462502 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:01.462443 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:01.463212 15237 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.464135 15278 ts_tablet_manager.cc:1434] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
W20260709 13:02:01.464275 15128 leader_election.cc:343] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.453019 15279 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.465283 15278 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.469738 15277 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.470798 15237 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:01.473212 15126 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:01.474289 15278 tablet_bootstrap.cc:654] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.476215 15279 tablet_bootstrap.cc:654] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.477205 15277 tablet_bootstrap.cc:654] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.481886 15284 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.484257 15284 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:01.485759 15284 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.487347 15277 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.487599 15278 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.487901 15277 ts_tablet_manager.cc:1403] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:02:01.488058 15278 ts_tablet_manager.cc:1403] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 13:02:01.488688 15279 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.489295 15279 ts_tablet_manager.cc:1403] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.006s
I20260709 13:02:01.491158 15277 raft_consensus.cc:359] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.491386 15278 raft_consensus.cc:359] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.492040 15277 raft_consensus.cc:385] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.492197 15278 raft_consensus.cc:385] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.492421 15277 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.492631 15278 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.493342 15277 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.493801 15278 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.493476 15279 raft_consensus.cc:359] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.495093 15279 raft_consensus.cc:385] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.495561 15279 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.496676 15278 ts_tablet_manager.cc:1434] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:01.496474 15279 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.497860 15278 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.501878 15279 ts_tablet_manager.cc:1434] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:02:01.503026 15279 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.503556 14985 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.504753 15278 tablet_bootstrap.cc:654] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.505313 15308 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.505801 15308 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.508926 15308 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.509804 15279 tablet_bootstrap.cc:654] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.510308 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.510989 15088 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.511425 15277 ts_tablet_manager.cc:1434] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:02:01.512421 15128 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:01.515530 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:01.517666 15126 leader_election.cc:343] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.518975 15308 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.519506 15308 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.519927 15308 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.523161 15279 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.523713 15279 ts_tablet_manager.cc:1403] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:02:01.526641 15277 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.526861 15279 raft_consensus.cc:359] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.531702 15279 raft_consensus.cc:385] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.532549 15279 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.530530 15308 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.533900 15279 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.534808 15277 tablet_bootstrap.cc:654] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.536360 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:01.537036 15088 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.538519 15279 ts_tablet_manager.cc:1434] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:02:01.539877 15279 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.541486 15284 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.541935 15284 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.546209 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.544997 15284 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.549876 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.550534 15237 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
W20260709 13:02:01.551848 15128 leader_election.cc:343] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.553407 15126 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:01.555421 15284 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.555902 15284 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.556347 15284 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.560305 15279 tablet_bootstrap.cc:654] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.566386 15088 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:01.568192 15128 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:01.570389 15314 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.571947 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:02:01.573833 15126 leader_election.cc:343] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.574102 15314 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:01.575742 15279 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.576225 15279 ts_tablet_manager.cc:1403] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.001s
I20260709 13:02:01.575551 15314 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.577688 15278 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.578212 15278 ts_tablet_manager.cc:1403] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.081s	user 0.021s	sys 0.028s
I20260709 13:02:01.574553 15308 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.579246 15279 raft_consensus.cc:359] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.579998 15279 raft_consensus.cc:385] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.580298 15279 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.581526 15278 raft_consensus.cc:359] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.582572 15278 raft_consensus.cc:385] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.582998 15278 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.584200 15278 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.587402 15278 ts_tablet_manager.cc:1434] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:02:01.584746 15284 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.587599 15279 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.588954 15278 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.590018 15279 ts_tablet_manager.cc:1434] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:02:01.593905 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:02:01.595808 15128 leader_election.cc:343] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.598110 15278 tablet_bootstrap.cc:654] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.603415 15284 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.603986 15277 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.604480 15277 ts_tablet_manager.cc:1403] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.078s	user 0.021s	sys 0.041s
I20260709 13:02:01.604737 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:01.605381 15237 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.607883 15279 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.608192 15277 raft_consensus.cc:359] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.608994 15277 raft_consensus.cc:385] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.609299 15277 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.610414 15277 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.615178 15277 ts_tablet_manager.cc:1434] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:02:01.615368 15279 tablet_bootstrap.cc:654] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.618988 15277 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.619495 15237 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:01.617712 15278 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.620719 15126 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:01.621313 15278 ts_tablet_manager.cc:1403] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.007s
I20260709 13:02:01.622696 15285 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.623188 15285 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.624545 15278 raft_consensus.cc:359] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.625293 15278 raft_consensus.cc:385] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.625792 15278 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.626166 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.626777 15088 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.626622 15278 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.627465 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.627753 15284 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.628079 15202 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.628264 15284 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:01.625329 15285 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.629165 15284 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:01.629278 15203 leader_election.cc:343] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.629891 15285 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.630259 15285 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.630602 15285 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.634215 15279 tablet_bootstrap.cc:492] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.634714 15279 ts_tablet_manager.cc:1403] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.006s
I20260709 13:02:01.635135 15278 ts_tablet_manager.cc:1434] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:02:01.636170 15278 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.637363 15279 raft_consensus.cc:359] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.638010 15279 raft_consensus.cc:385] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.638280 15279 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.638911 15279 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.641031 15279 ts_tablet_manager.cc:1434] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:02:01.642169 15277 tablet_bootstrap.cc:654] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.646477 15279 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.647120 15285 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.650667 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:01.651480 15278 tablet_bootstrap.cc:654] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.651422 15088 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.657097 15278 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.657259 15279 tablet_bootstrap.cc:654] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.657626 15278 ts_tablet_manager.cc:1403] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.000s
I20260709 13:02:01.659291 15088 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:01.660481 15202 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.660712 15278 raft_consensus.cc:359] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.661520 15323 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.661755 15278 raft_consensus.cc:385] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.662334 15278 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.663209 15278 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.665794 15285 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.665795 15278 ts_tablet_manager.cc:1434] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:01.666419 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:01.667631 15278 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
W20260709 13:02:01.667900 15203 leader_election.cc:343] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.668889 15277 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.669488 15277 ts_tablet_manager.cc:1403] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.000s
I20260709 13:02:01.671077 14985 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.675925 15278 tablet_bootstrap.cc:654] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.676548 14985 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.673629 15277 raft_consensus.cc:359] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.678013 15277 raft_consensus.cc:385] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.678507 15277 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.679628 15277 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.682936 15277 ts_tablet_manager.cc:1434] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:02:01.685148 15277 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.694053 15277 tablet_bootstrap.cc:654] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.694413 15323 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:01.695458 15323 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.708223 15279 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.708777 15279 ts_tablet_manager.cc:1403] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.024s
I20260709 13:02:01.710692 15277 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.710855 15278 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.711180 15277 ts_tablet_manager.cc:1403] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.003s
I20260709 13:02:01.711333 15278 ts_tablet_manager.cc:1403] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.044s	user 0.028s	sys 0.010s
I20260709 13:02:01.711807 15279 raft_consensus.cc:359] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.712534 15279 raft_consensus.cc:385] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.712874 15279 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.713802 15279 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.713969 15277 raft_consensus.cc:359] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.714664 15277 raft_consensus.cc:385] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.714963 15277 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.715576 15278 raft_consensus.cc:359] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.715714 15277 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.716301 15278 raft_consensus.cc:385] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.715806 14986 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.716909 15279 ts_tablet_manager.cc:1434] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:01.721261 15279 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.715843 15323 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.724694 15323 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.726855 15283 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.727102 15323 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.716656 15278 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.727442 15279 tablet_bootstrap.cc:654] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.727372 15283 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.727986 15278 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.729530 15283 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.730400 15278 ts_tablet_manager.cc:1434] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:02:01.730466 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.731388 15278 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.731442 15163 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.718255 15277 ts_tablet_manager.cc:1434] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:02:01.730661 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.733366 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.734139 15088 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.734246 15237 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:01.735679 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.736927 15323 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.737443 15323 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.737951 15278 tablet_bootstrap.cc:654] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.733415 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.739135 15162 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:01.739456 15277 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.740679 15279 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.740516 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:01.741153 15279 ts_tablet_manager.cc:1403] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.005s
I20260709 13:02:01.741436 15283 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.741820 15283 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.742121 15283 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.743798 15279 raft_consensus.cc:359] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.744478 15279 raft_consensus.cc:385] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.744735 15279 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.745640 15279 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.737883 15323 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.746932 15277 tablet_bootstrap.cc:654] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.749425 15279 ts_tablet_manager.cc:1434] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:01.750524 15279 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.757494 15323 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.761480 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:01.762241 15162 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.777668 15279 tablet_bootstrap.cc:654] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.784068 15278 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.784818 15278 ts_tablet_manager.cc:1403] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.054s	user 0.030s	sys 0.021s
I20260709 13:02:01.760329 15283 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.773379 15323 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.790823 15162 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:01.792390 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:01.793104 15237 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.795116 15278 raft_consensus.cc:359] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.796048 15278 raft_consensus.cc:385] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.796396 15278 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.797149 15283 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.797284 15278 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.798383 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:01.799679 15088 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.799885 15283 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.800755 15278 ts_tablet_manager.cc:1434] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:02:01.800511 15283 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.801735 15278 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.803332 15283 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.804112 15334 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.804592 15334 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.804774 15203 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.808911 15278 tablet_bootstrap.cc:654] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.808655 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.810813 15236 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:01.811425 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:01.812158 15162 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.809334 15323 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.814090 15323 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:01.814934 15323 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.818084 15334 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.822808 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:01.828613 15051 leader_election.cc:343] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.828924 15162 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:01.829290 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.831907 15088 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:01.832803 15237 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:01.834587 15051 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.835644 15279 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:01.836117 15279 ts_tablet_manager.cc:1403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.011s
I20260709 13:02:01.836226 15334 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:01.837002 15277 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.837755 15277 ts_tablet_manager.cc:1403] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.099s	user 0.032s	sys 0.023s
I20260709 13:02:01.834594 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.838714 15163 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:01.836975 15334 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:01.839696 15334 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.839751 15279 raft_consensus.cc:359] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.840755 15279 raft_consensus.cc:385] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.840757 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:01.841337 15279 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.842715 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:01.842651 15279 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.843648 15283 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.844334 15336 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.844719 15336 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.845085 15336 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.847221 15277 raft_consensus.cc:359] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.848148 15277 raft_consensus.cc:385] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.848488 15277 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.845944 15279 ts_tablet_manager.cc:1434] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:02:01.849395 15277 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.852990 15279 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:01.858795 15283 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:01.859053 15278 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:01.859144 15336 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.859640 15278 ts_tablet_manager.cc:1403] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.019s
I20260709 13:02:01.862172 14986 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.864666 15278 raft_consensus.cc:359] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.865747 15278 raft_consensus.cc:385] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.866139 15278 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.867043 15277 ts_tablet_manager.cc:1434] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 13:02:01.868755 15277 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:01.871276 15279 tablet_bootstrap.cc:654] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.867066 15278 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.873152 15278 ts_tablet_manager.cc:1434] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:02:01.876574 15277 tablet_bootstrap.cc:654] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.876812 15278 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:01.886307 15278 tablet_bootstrap.cc:654] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.888319 15277 tablet_bootstrap.cc:492] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:01.888888 15277 ts_tablet_manager.cc:1403] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:02:01.891656 15277 raft_consensus.cc:359] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.892493 15277 raft_consensus.cc:385] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:01.892943 15277 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:01.893743 15277 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.896626 15336 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.898473 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:01.899034 15163 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.899408 15277 ts_tablet_manager.cc:1434] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:02:01.899874 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:01.900312 15277 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
W20260709 13:02:01.901156 15051 leader_election.cc:343] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.907038 15277 tablet_bootstrap.cc:654] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:01.902139 15323 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.907866 15323 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.911857 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.912077 15284 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.912529 15088 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.912528 15284 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.914134 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.914827 15284 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.915120 15328 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.915499 15328 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.915887 15328 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.916606 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.917513 15088 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.919211 15128 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:01.920356 15284 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.920866 15284 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.921291 15284 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.922770 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.923648 15237 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.926546 15323 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.927536 15314 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.928434 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.930843 15314 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.931201 15162 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.934572 15314 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:01.937933 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:01.938899 15237 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.940059 15328 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.940629 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:01.941685 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:01.942570 15088 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:01.942782 15314 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.943472 15314 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.943964 15314 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.946529 15323 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.947294 15323 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.948024 14983 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.953366 15087 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.951975 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:01.963261 15202 leader_election.cc:343] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:01.963454 15323 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.965384 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:01.966337 15162 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:01.968160 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:01.970038 15323 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:01.970759 15323 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.971289 15323 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.982883 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:01.930802 15284 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.996714 15162 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.001184 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:01.990973 15328 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:01.976493 15279 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.001941 15086 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.002413 15279 ts_tablet_manager.cc:1403] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.151s	user 0.030s	sys 0.010s
I20260709 13:02:02.003690 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:02.004673 15237 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.973372 15163 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:02.008747 15308 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.009230 15284 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.009076 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:02.011029 15336 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.011694 15336 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:02.011845 15308 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.012883 15336 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:01.998342 15314 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.021622 15086 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:02.022101 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:02.022930 15236 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.025365 15128 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:02.026659 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.027374 15085 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:01.972798 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.031879 15088 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.033077 15308 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.033603 15308 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:02.034443 15308 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.038065 14986 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.043288 15085 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:02.044694 15128 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:02.055872 15349 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.016615 15279 raft_consensus.cc:359] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.060163 15279 raft_consensus.cc:385] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.060411 15279 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.061151 15314 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.062217 15279 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.057958 15349 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.066025 15278 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.070807 15278 ts_tablet_manager.cc:1403] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.194s	user 0.051s	sys 0.078s
I20260709 13:02:02.066835 15279 ts_tablet_manager.cc:1434] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.001s
I20260709 13:02:02.057924 15236 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:02.074904 15279 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.091991 15237 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:02.093256 15328 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.093787 15328 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.096622 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.099334 15088 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.099542 15328 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.100654 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.101833 15314 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.103466 15277 tablet_bootstrap.cc:492] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.108649 15277 ts_tablet_manager.cc:1403] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.209s	user 0.047s	sys 0.103s
I20260709 13:02:02.108358 15278 raft_consensus.cc:359] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.106531 15323 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.109396 15278 raft_consensus.cc:385] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.113756 15278 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.117385 15278 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.118251 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:02.108723 15162 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.119145 15328 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.119791 15328 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:02.121246 15278 ts_tablet_manager.cc:1434] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:02:02.122252 15278 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.116448 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:02.122864 15285 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.120796 15328 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.124316 15277 raft_consensus.cc:359] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.125012 15277 raft_consensus.cc:385] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.125278 15277 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.127027 15308 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.127503 15308 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.129626 15308 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.131175 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:02.132066 15323 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.131999 15236 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:02.132707 15277 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.134936 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.135718 15277 ts_tablet_manager.cc:1434] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.000s
I20260709 13:02:02.137013 15277 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
W20260709 13:02:02.137633 15202 leader_election.cc:343] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.141424 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:02.143150 15308 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.143571 15308 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.142310 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.144150 15162 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.144595 15308 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.139925 14986 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.146420 15349 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:02.151371 15349 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.153671 15279 tablet_bootstrap.cc:654] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.147753 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.159977 15323 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.160506 15323 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.173316 15162 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.178017 15308 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.178777 15236 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.180197 15323 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.184582 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.184762 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:02:02.188223 15203 leader_election.cc:343] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.189795 15323 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.190310 15323 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:02.184948 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.191205 15323 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:02.195153 15202 leader_election.cc:343] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.195848 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:02.197038 15328 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.201380 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.202818 15308 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.181340 15088 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:02.207974 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:02.208714 15234 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.224143 15354 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:02:02.224527 15308 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.225206 15277 tablet_bootstrap.cc:654] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.236402 15278 tablet_bootstrap.cc:654] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.222901 15349 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:02:02.240672 14986 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.225063 15308 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.239981 15234 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:02.251278 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:02.252563 15349 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.253244 15349 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:02.259326 15349 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.260866 15279 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.261399 15279 ts_tablet_manager.cc:1403] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.189s	user 0.003s	sys 0.014s
I20260709 13:02:02.262861 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.263744 15088 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:02:02.268254 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:02.268779 15279 raft_consensus.cc:359] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.269580 15279 raft_consensus.cc:385] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.269971 15279 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.270694 15279 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.272241 15308 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.274034 15279 ts_tablet_manager.cc:1434] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:02:02.273773 15328 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.274470 15163 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.275038 15279 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.279066 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:02.284317 15126 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:02.287189 14985 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.289314 15278 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.289901 15278 ts_tablet_manager.cc:1403] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.168s	user 0.018s	sys 0.015s
I20260709 13:02:02.296891 15323 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.297824 15328 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.298378 15328 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.300918 15328 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.303748 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.308283 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.314316 15349 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.314141 15328 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:02.314604 15202 leader_election.cc:343] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.314010 15278 raft_consensus.cc:359] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.317596 15278 raft_consensus.cc:385] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.317951 15278 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.318634 15278 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.320868 15278 ts_tablet_manager.cc:1434] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.000s
I20260709 13:02:02.326438 15161 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:02.327739 15277 tablet_bootstrap.cc:492] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.328426 15277 ts_tablet_manager.cc:1403] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.192s	user 0.018s	sys 0.001s
I20260709 13:02:02.330448 15279 tablet_bootstrap.cc:654] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.338467 15277 raft_consensus.cc:359] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.340472 15277 raft_consensus.cc:385] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.341231 15278 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.345990 15328 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:02:02.347939 15278 tablet_bootstrap.cc:654] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.351522 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:02.350283 14985 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.355062 15323 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.355520 15323 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.355938 15323 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.357064 15277 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.357829 15277 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.360157 15277 ts_tablet_manager.cc:1434] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.000s
I20260709 13:02:02.378446 15277 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.382613 15323 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.389686 15277 tablet_bootstrap.cc:654] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.401538 14986 catalog_manager.cc:5697] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.403548 15334 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.399711 15279 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.407374 15279 ts_tablet_manager.cc:1403] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.133s	user 0.021s	sys 0.027s
I20260709 13:02:02.409159 15277 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.409622 15277 ts_tablet_manager.cc:1403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.002s
I20260709 13:02:02.412770 15277 raft_consensus.cc:359] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.413678 15277 raft_consensus.cc:385] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.413951 15277 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.414876 15323 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.414762 15277 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.416203 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.419070 15279 raft_consensus.cc:359] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.419765 15279 raft_consensus.cc:385] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.419852 15163 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.420085 15279 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.420697 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.423009 15277 ts_tablet_manager.cc:1434] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
W20260709 13:02:02.423204 15202 leader_election.cc:343] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.424008 15277 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.420790 15279 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.429667 15279 ts_tablet_manager.cc:1434] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:02:02.429849 15163 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.430781 15277 tablet_bootstrap.cc:654] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.430812 15279 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.431416 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:02.430356 15341 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.441018 15279 tablet_bootstrap.cc:654] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.441138 15277 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.441684 15277 ts_tablet_manager.cc:1403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.001s
I20260709 13:02:02.436000 15278 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.442574 15278 ts_tablet_manager.cc:1403] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.102s	user 0.007s	sys 0.005s
I20260709 13:02:02.444568 15277 raft_consensus.cc:359] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.445688 15277 raft_consensus.cc:385] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.445996 15277 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.446844 15277 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.448691 15277 ts_tablet_manager.cc:1434] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:02:02.452401 15277 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.453648 15375 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.454293 15375 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:02.455118 15375 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.456194 15328 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.456966 15328 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.460332 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.460721 15328 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.466586 15279 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.466571 15278 raft_consensus.cc:359] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.467262 15279 ts_tablet_manager.cc:1403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.005s
I20260709 13:02:02.467397 15278 raft_consensus.cc:385] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.469161 15278 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.470909 15278 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.472314 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.473346 15162 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:02.473826 15278 ts_tablet_manager.cc:1434] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:02:02.475443 15278 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.476118 15203 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:02.477954 15376 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.496286 15279 raft_consensus.cc:359] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.497126 15279 raft_consensus.cc:385] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.497414 15279 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.498167 15279 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.500317 15279 ts_tablet_manager.cc:1434] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 13:02:02.501528 15277 tablet_bootstrap.cc:654] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.501401 15341 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.501207 15279 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.508526 15278 tablet_bootstrap.cc:654] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.494798 15376 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.518723 14985 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.522599 15334 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.525491 15279 tablet_bootstrap.cc:654] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.535902 15279 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.536391 15279 ts_tablet_manager.cc:1403] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:02:02.540546 15279 raft_consensus.cc:359] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.541345 15279 raft_consensus.cc:385] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.541637 15279 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.540557 15277 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.542654 15277 ts_tablet_manager.cc:1403] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.007s
I20260709 13:02:02.542428 15279 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.544553 15279 ts_tablet_manager.cc:1434] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:02.545678 15279 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.545614 15277 raft_consensus.cc:359] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.546424 15277 raft_consensus.cc:385] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.546711 15277 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.551199 15277 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.552472 15279 tablet_bootstrap.cc:654] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.554119 15277 ts_tablet_manager.cc:1434] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:02:02.556411 15277 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.558547 15278 tablet_bootstrap.cc:492] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.559051 15278 ts_tablet_manager.cc:1403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.018s
I20260709 13:02:02.562137 15277 tablet_bootstrap.cc:654] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.570089 15278 raft_consensus.cc:359] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.570820 15278 raft_consensus.cc:385] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.571094 15278 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.571759 15278 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.580360 15278 ts_tablet_manager.cc:1434] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:02:02.580508 15376 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.581225 15336 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.581864 15336 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.583727 15277 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.584206 15277 ts_tablet_manager.cc:1403] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.000s
I20260709 13:02:02.584508 15336 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.586560 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.587011 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:02.587267 15163 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:02.587141 15277 raft_consensus.cc:359] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.587723 15235 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:02.587884 15277 raft_consensus.cc:385] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.588322 15277 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.588688 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:02.589205 15277 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.590075 15336 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.590749 15336 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.591147 15336 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.590839 15278 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.591708 15277 ts_tablet_manager.cc:1434] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:02:02.592762 15277 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.598726 15277 tablet_bootstrap.cc:654] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.603775 15279 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.603861 15354 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.604521 15279 ts_tablet_manager.cc:1403] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.016s
I20260709 13:02:02.606941 15376 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.609724 15279 raft_consensus.cc:359] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.610728 15279 raft_consensus.cc:385] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.611148 15279 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.606642 15336 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.613219 15279 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.617151 15279 ts_tablet_manager.cc:1434] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:02:02.618119 15279 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.619318 15354 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.622443 15336 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.622874 15085 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.624449 15278 tablet_bootstrap.cc:654] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.624534 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.625363 15162 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.625303 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:02.625982 15234 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.635584 15162 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:02.637048 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:02.639537 15334 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.640244 15334 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:02.641372 15334 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.651947 15279 tablet_bootstrap.cc:654] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.645978 15234 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:02.661550 15376 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.670125 15336 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.670866 15336 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.674503 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:02.676339 15054 leader_election.cc:343] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.677341 15277 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.677852 15336 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.677870 15277 ts_tablet_manager.cc:1403] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.000s
I20260709 13:02:02.688948 15373 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.689965 15336 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.690685 15373 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.690963 15336 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.694634 15373 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.695327 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.691430 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:02.696796 15235 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:02.699347 15084 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:02.706809 15054 leader_election.cc:343] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.705857 15277 raft_consensus.cc:359] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.709604 15277 raft_consensus.cc:385] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.710026 15277 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.711367 15277 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.708609 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.721669 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:02.717069 15336 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.725375 15277 ts_tablet_manager.cc:1434] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 13:02:02.729553 15278 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.730057 15278 ts_tablet_manager.cc:1403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.139s	user 0.050s	sys 0.010s
I20260709 13:02:02.729856 15277 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.730391 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.735596 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:02:02.737708 15051 leader_election.cc:343] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.735778 15162 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:02.738186 15051 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:02.747196 15278 raft_consensus.cc:359] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.754343 15278 raft_consensus.cc:385] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.754733 15278 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.755326 15336 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.757575 15308 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.750717 15279 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.751952 15328 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.752622 15203 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:02.740231 15334 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.761059 15323 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.761183 15334 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.761487 15323 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.761719 15334 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.761871 15323 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.765707 15278 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:02.766597 15202 leader_election.cc:343] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.769778 15328 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.770341 15328 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.772783 15323 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.780514 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:02.782048 15088 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:02.776790 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.778370 15308 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:02:02.775638 15279 ts_tablet_manager.cc:1403] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.143s	user 0.022s	sys 0.041s
I20260709 13:02:02.772568 15328 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.786453 15202 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:02.780599 15323 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.787940 15323 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:02.775616 15334 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.788489 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.788954 15323 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.789301 15163 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.789469 15323 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.791342 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.791985 15162 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.788321 15279 raft_consensus.cc:359] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.797282 15279 raft_consensus.cc:385] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:02:02.797449 15203 leader_election.cc:343] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.788384 15278 ts_tablet_manager.cc:1434] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.004s
I20260709 13:02:02.798506 15334 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:02.797644 15279 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.798861 15278 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.799306 15354 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:02.799528 15279 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.801554 15279 ts_tablet_manager.cc:1434] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:02:02.802560 15279 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:02.782686 14983 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.805141 15278 tablet_bootstrap.cc:654] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.777432 15277 tablet_bootstrap.cc:654] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.807847 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:02.808434 15234 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.810401 15163 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.811892 15203 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:02.797755 15323 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.813977 15373 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.815935 15373 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:02.816900 15373 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.808254 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:02:02.820372 15202 leader_election.cc:343] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.834286 15323 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.834807 15278 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:02.835366 15278 ts_tablet_manager.cc:1403] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.006s
I20260709 13:02:02.838526 15279 tablet_bootstrap.cc:654] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.872967 15162 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:02.874732 15234 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:02.876469 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:02.839121 15278 raft_consensus.cc:359] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.878489 15084 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:02.879150 15278 raft_consensus.cc:385] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.879602 15278 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.880647 15278 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.883836 15278 ts_tablet_manager.cc:1434] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.005s
I20260709 13:02:02.886098 15278 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:02.887964 15277 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:02.879622 15054 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:02.889259 15373 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.890045 15373 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.890861 15334 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.891748 15334 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:02.893563 15334 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.896375 15373 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:02.895035 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:02.899765 15202 leader_election.cc:343] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.901067 15278 tablet_bootstrap.cc:654] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:02.902511 15277 ts_tablet_manager.cc:1403] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.007s
I20260709 13:02:02.904284 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:02.904634 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:02.905057 15308 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:02:02.906092 15203 leader_election.cc:343] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:02.910387 15203 leader_election.cc:343] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:02.911710 15277 raft_consensus.cc:359] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.912591 15277 raft_consensus.cc:385] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.913048 15277 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.912514 15203 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:02.914808 15323 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:02.915778 15084 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:02.916899 15277 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.922680 15202 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:02.926692 15328 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:02.927414 15328 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:02.928925 15328 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.955046 15376 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:02.972195 15277 ts_tablet_manager.cc:1434] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:02:02.979681 15279 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:02.980459 15279 ts_tablet_manager.cc:1403] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.178s	user 0.020s	sys 0.021s
I20260709 13:02:02.981552 15277 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:02.984277 15279 raft_consensus.cc:359] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.985275 15279 raft_consensus.cc:385] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:02.985548 15279 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:02.987507 15279 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:02.997377 15279 ts_tablet_manager.cc:1434] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.017s	user 0.017s	sys 0.000s
I20260709 13:02:03.015797 15277 tablet_bootstrap.cc:654] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.033695 14983 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.041337 15279 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.048965 15334 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.049631 15334 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.052100 15334 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.052631 15308 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.053206 15308 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.061296 15375 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.061906 15375 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.066207 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:03.064726 15308 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.064976 15375 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:03.080961 15128 leader_election.cc:343] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.064271 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.081322 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:03.100044 15054 leader_election.cc:343] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:03.102418 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:03.103389 15202 leader_election.cc:343] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.123179 15278 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.123732 15278 ts_tablet_manager.cc:1403] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.238s	user 0.019s	sys 0.027s
I20260709 13:02:03.125111 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.130532 15277 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.130926 15278 raft_consensus.cc:359] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.131608 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.132469 15278 raft_consensus.cc:385] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.132797 15278 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.132859 15277 ts_tablet_manager.cc:1403] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.003s
I20260709 13:02:03.134513 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:03.135105 15126 leader_election.cc:343] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.135619 15126 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:02:03.138757 15051 leader_election.cc:343] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.139281 15277 raft_consensus.cc:359] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.140049 15277 raft_consensus.cc:385] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.140318 15277 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.141409 15354 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.139285 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.143878 14985 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.149461 14985 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.143980 15336 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.147208 15279 tablet_bootstrap.cc:654] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.144593 15278 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.153362 15162 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:03.153162 15277 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.155495 15277 ts_tablet_manager.cc:1434] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:02:03.157074 15277 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.157301 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:03.162499 15417 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.163013 15417 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.163419 15417 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.165670 15278 ts_tablet_manager.cc:1434] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:02:03.172259 15417 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.174585 15278 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.175582 15279 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.176038 15279 ts_tablet_manager.cc:1403] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.016s
I20260709 13:02:03.176373 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:02:03.178073 15202 leader_election.cc:343] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.179100 15417 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:03.178855 15279 raft_consensus.cc:359] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.181334 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:03.181993 15162 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.186749 15277 tablet_bootstrap.cc:654] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.179629 15279 raft_consensus.cc:385] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.187160 15279 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.187868 15279 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.190346 15279 ts_tablet_manager.cc:1434] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:02:03.191489 15279 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.196314 15278 tablet_bootstrap.cc:654] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.192035 15162 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:03.200467 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:03.201797 15336 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.202178 15277 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.202314 15336 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.202858 15277 ts_tablet_manager.cc:1403] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.003s
I20260709 13:02:03.205082 15376 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.205677 15376 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:03.206568 15376 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.206580 15277 raft_consensus.cc:359] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.207583 15277 raft_consensus.cc:385] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.207463 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.209295 15336 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.212311 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.213248 15162 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:03.211650 15279 tablet_bootstrap.cc:654] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.221227 15051 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.222347 15336 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.223533 15419 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.228025 15277 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.229393 15277 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.232769 15277 ts_tablet_manager.cc:1434] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.003s
I20260709 13:02:03.234138 15277 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.241283 15277 tablet_bootstrap.cc:654] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.251004 15419 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:03.248766 15336 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.257562 15277 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.258071 15277 ts_tablet_manager.cc:1403] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.012s
I20260709 13:02:03.260108 15376 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:03.261056 15277 raft_consensus.cc:359] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.261746 15277 raft_consensus.cc:385] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.263213 15279 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.263597 15277 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.263859 15279 ts_tablet_manager.cc:1403] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.005s
I20260709 13:02:03.264328 15277 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.266637 15279 raft_consensus.cc:359] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.267362 15279 raft_consensus.cc:385] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.267639 15279 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.267704 15277 ts_tablet_manager.cc:1434] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:02:03.268775 15278 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.268380 15279 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.269279 15278 ts_tablet_manager.cc:1403] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.095s	user 0.034s	sys 0.021s
I20260709 13:02:03.270891 15279 ts_tablet_manager.cc:1434] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:02:03.271763 15277 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.272027 15279 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.272468 15278 raft_consensus.cc:359] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.273769 15336 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.278100 15278 raft_consensus.cc:385] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.278379 15277 tablet_bootstrap.cc:654] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.278302 15336 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.279975 14984 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.281102 15278 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.281525 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.282238 15279 tablet_bootstrap.cc:654] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.281909 15278 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.282162 15162 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.284003 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.284440 15278 ts_tablet_manager.cc:1434] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:02:03.285056 15419 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.285486 15419 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.285830 15419 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.286041 15278 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.295302 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.296324 15235 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.299243 15336 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.300910 15278 tablet_bootstrap.cc:654] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.302000 15419 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.317070 15418 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.320640 15354 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.321389 15354 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.324363 15354 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.326670 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:03.327555 15086 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:03.333086 15128 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.334589 15349 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.335008 15349 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.335418 15349 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.341498 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.342381 15235 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:03.349294 15278 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.349850 15278 ts_tablet_manager.cc:1403] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.022s
I20260709 13:02:03.359629 15278 raft_consensus.cc:359] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.360380 15278 raft_consensus.cc:385] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.360654 15278 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.361342 15278 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.363382 15278 ts_tablet_manager.cc:1434] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:02:03.364508 15278 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.371551 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:03.372274 15162 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.381610 15278 tablet_bootstrap.cc:654] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.386286 15334 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.387260 15419 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.389483 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.390152 15237 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.389611 15349 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.419818 15349 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.422387 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.423084 15162 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.423548 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:03.424132 15084 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.427933 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.428833 15336 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.429016 15237 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.429365 15336 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:03.430219 15336 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.423130 15236 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.438874 15279 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.439465 15279 ts_tablet_manager.cc:1403] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.168s	user 0.050s	sys 0.070s
I20260709 13:02:03.447005 15236 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:03.449173 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:03.450054 15349 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.450129 15279 raft_consensus.cc:359] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.450601 15349 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:03.450867 15279 raft_consensus.cc:385] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.451187 15279 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.451433 15349 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.451846 15279 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.453970 15279 ts_tablet_manager.cc:1434] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:02:03.455072 15279 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.458634 15278 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.458634 15277 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.459220 15278 ts_tablet_manager.cc:1403] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.095s	user 0.014s	sys 0.005s
I20260709 13:02:03.459446 15277 ts_tablet_manager.cc:1403] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.191s	user 0.046s	sys 0.025s
I20260709 13:02:03.458742 15419 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.461898 15278 raft_consensus.cc:359] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.462627 15278 raft_consensus.cc:385] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.462903 15278 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.463546 15278 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.465847 15277 raft_consensus.cc:359] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.466434 15278 ts_tablet_manager.cc:1434] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:02:03.465691 14984 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.473879 15278 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.469780 15418 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:03.480350 15278 tablet_bootstrap.cc:654] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.470999 15279 tablet_bootstrap.cc:654] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.483626 15162 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.483932 15277 raft_consensus.cc:385] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.484269 15277 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.484958 15277 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.487114 15277 ts_tablet_manager.cc:1434] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.000s
I20260709 13:02:03.488067 15277 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.490392 15084 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
W20260709 13:02:03.501691 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:03.506281 15277 tablet_bootstrap.cc:654] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.506188 15418 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.517604 15279 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.518251 15279 ts_tablet_manager.cc:1403] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.007s
I20260709 13:02:03.528987 14985 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.536319 15336 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.536901 15336 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.539003 15336 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.541544 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.543176 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.543996 15162 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:03.545025 15279 raft_consensus.cc:359] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.545946 15279 raft_consensus.cc:385] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.546298 15279 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.546885 15054 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.547178 15279 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.547902 15336 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.551371 15278 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.551935 15278 ts_tablet_manager.cc:1403] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.078s	user 0.020s	sys 0.024s
I20260709 13:02:03.555902 15278 raft_consensus.cc:359] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.556851 15278 raft_consensus.cc:385] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.557214 15278 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.558264 15278 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.560920 15308 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.561457 15308 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.562592 15279 ts_tablet_manager.cc:1434] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.044s	user 0.010s	sys 0.000s
I20260709 13:02:03.564016 15279 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.564113 15308 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.565371 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:03.566151 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:03.566915 15128 leader_election.cc:343] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:03.571854 15126 leader_election.cc:343] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.572336 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.573262 15279 tablet_bootstrap.cc:654] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.560967 15278 ts_tablet_manager.cc:1434] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:02:03.576561 15278 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.577073 15308 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.578162 15336 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.583129 15277 tablet_bootstrap.cc:492] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.583642 15277 ts_tablet_manager.cc:1403] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.005s
I20260709 13:02:03.586213 15278 tablet_bootstrap.cc:654] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.588073 15277 raft_consensus.cc:359] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.588871 15277 raft_consensus.cc:385] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.589144 15277 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.589217 15336 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.589892 15336 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.589905 15277 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.592573 15277 ts_tablet_manager.cc:1434] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:02:03.593709 15277 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.599546 15277 tablet_bootstrap.cc:654] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.601225 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:03.602587 15054 leader_election.cc:343] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.613440 15277 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.613991 15277 ts_tablet_manager.cc:1403] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:02:03.614821 15279 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.615295 15279 ts_tablet_manager.cc:1403] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.004s
I20260709 13:02:03.616128 15278 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.616640 15278 ts_tablet_manager.cc:1403] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.015s
I20260709 13:02:03.617231 15277 raft_consensus.cc:359] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.617987 15277 raft_consensus.cc:385] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.618325 15277 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.618063 15279 raft_consensus.cc:359] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.618804 15279 raft_consensus.cc:385] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.619081 15279 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.619086 15277 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.619470 15278 raft_consensus.cc:359] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.619801 15279 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.620217 15278 raft_consensus.cc:385] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.620996 15278 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.621939 15279 ts_tablet_manager.cc:1434] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:02:03.621767 15278 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.622198 15308 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.623070 15279 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.622710 15308 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.624455 15278 ts_tablet_manager.cc:1434] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:02:03.626519 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:03.627154 15088 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:03.628813 15128 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.629694 15349 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.629804 15279 tablet_bootstrap.cc:654] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.630125 15349 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.630542 15349 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.624823 15308 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.634063 15278 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.638532 15349 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.641294 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.641547 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:03.642263 15088 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.643590 15235 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:03.647924 15336 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.649207 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.649927 15235 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.651120 15088 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:03.651517 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:03.652979 15336 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.653183 15128 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.653364 15336 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.653489 15279 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.653831 15336 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.654074 15308 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.621920 15277 ts_tablet_manager.cc:1434] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:02:03.654799 15308 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:03.655505 15277 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.655941 15278 tablet_bootstrap.cc:654] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.655685 15308 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.657943 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.658533 15235 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.661827 15277 tablet_bootstrap.cc:654] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.664955 15349 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.667804 15336 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.670172 15235 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:03.654045 15279 ts_tablet_manager.cc:1403] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 13:02:03.674566 15278 tablet_bootstrap.cc:492] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.674489 15279 raft_consensus.cc:359] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.675091 15278 ts_tablet_manager.cc:1403] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.003s
I20260709 13:02:03.675304 15279 raft_consensus.cc:385] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.675691 15277 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.676270 15277 ts_tablet_manager.cc:1403] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.003s
I20260709 13:02:03.677829 15278 raft_consensus.cc:359] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.678164 15419 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.678469 15278 raft_consensus.cc:385] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.678781 15278 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.678695 15419 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.679214 15277 raft_consensus.cc:359] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.679912 15277 raft_consensus.cc:385] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.679711 15278 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.680222 15277 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.681169 15277 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.681880 15278 ts_tablet_manager.cc:1434] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:02:03.682962 15278 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.683218 15277 ts_tablet_manager.cc:1434] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:02:03.683749 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.675731 15279 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.684458 15277 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.684993 15279 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.686249 15418 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:03.686986 15279 ts_tablet_manager.cc:1434] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:02:03.688225 15279 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.672320 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:03.690142 15334 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.690667 15334 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.693081 15334 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:03.694208 15054 leader_election.cc:343] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.683785 14984 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.695406 15279 tablet_bootstrap.cc:654] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.695598 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.696295 15161 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.697293 15419 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.697851 15054 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.697992 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.698899 15235 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.700438 15277 tablet_bootstrap.cc:654] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.701277 15419 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.701887 15419 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.702361 15419 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.703544 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.706954 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.712100 15277 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.712616 15277 ts_tablet_manager.cc:1403] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.007s
I20260709 13:02:03.713661 15334 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.715535 15277 raft_consensus.cc:359] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.716200 15277 raft_consensus.cc:385] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.716475 15277 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.717072 15277 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.718681 15277 ts_tablet_manager.cc:1434] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:02:03.719698 15277 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.727888 15278 tablet_bootstrap.cc:654] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.727988 15277 tablet_bootstrap.cc:654] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.742105 15418 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:03.747120 15278 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:03.747852 15278 ts_tablet_manager.cc:1403] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.000s
I20260709 13:02:03.752059 15417 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.752594 15417 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.753508 15278 raft_consensus.cc:359] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.754559 15278 raft_consensus.cc:385] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.754941 15278 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.756093 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.755851 15278 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:03.757589 15203 leader_election.cc:343] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.758855 15278 ts_tablet_manager.cc:1434] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:02:03.760037 15278 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:03.759852 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:03.761425 15417 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:03.762161 15202 leader_election.cc:343] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.762620 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:03.764112 15418 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:03.789048 15336 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.792155 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.792821 15235 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.797175 15336 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.797766 15336 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.799891 15336 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.815927 15235 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.817863 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:03.864683 15277 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:03.869704 15277 ts_tablet_manager.cc:1403] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.150s	user 0.006s	sys 0.038s
I20260709 13:02:03.872721 15336 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:02:03.872507 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54785.9us
I20260709 13:02:03.873319 15336 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:03.866479 15308 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:02:03.874141 15336 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.875562 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.876307 15162 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:03.818204 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:03.879441 15279 tablet_bootstrap.cc:492] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:03.879946 15279 ts_tablet_manager.cc:1403] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.192s	user 0.037s	sys 0.033s
I20260709 13:02:03.879742 15419 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.874169 15277 raft_consensus.cc:359] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.881850 15277 raft_consensus.cc:385] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.882931 15277 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.883627 15277 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:03.884516 15051 leader_election.cc:343] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.885324 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:03.885934 15163 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.886173 15279 raft_consensus.cc:359] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.887421 15277 ts_tablet_manager.cc:1434] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:02:03.887841 15279 raft_consensus.cc:385] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:03.888207 15279 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:03.889097 15279 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.890105 15277 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:03.893639 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:03.894496 15435 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:03.895470 15435 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.895820 15435 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.895915 15163 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.896868 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.897451 15235 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.897516 15054 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.898422 15334 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.897804 15279 ts_tablet_manager.cc:1434] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:02:03.894891 15419 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.901268 15279 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:03.908654 15349 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:02:03.910657 15435 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.915977 15435 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:03.917419 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:03.917603 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:03.918553 15161 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.918532 15237 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.925760 15278 tablet_bootstrap.cc:654] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.905812 15235 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.922441 15334 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:03.932294 15088 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:03.926122 15336 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:02:03.943655 15161 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.945112 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:03.945969 15419 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:03.946635 15419 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:03.947180 15237 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:03.947700 15419 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.948769 15336 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:03.911824 14984 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.951570 15334 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.971558 15277 tablet_bootstrap.cc:654] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:03.975996 15419 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.977088 15419 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.988284 15417 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.988831 15417 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.990901 15417 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:03.991639 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:03.993214 15308 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.997968 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:03.998448 15054 leader_election.cc:343] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:03.997548 15308 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:03.998634 15084 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:04.000528 15308 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.001044 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.001195 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.001822 15084 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:04.001817 15417 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.002403 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.002761 15417 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.002892 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.003032 15163 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:04.003254 15417 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.003722 15308 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.004079 15308 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.004401 15308 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.011533 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.016319 15419 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.019104 15278 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.019816 15278 ts_tablet_manager.cc:1403] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.260s	user 0.014s	sys 0.055s
I20260709 13:02:04.020634 15279 tablet_bootstrap.cc:654] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.025429 15417 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.028702 15233 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:04.028954 15308 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.031041 15417 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:04.030908 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:04.031922 15084 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.038278 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:04.039234 15163 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:04.040630 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.044824 15278 raft_consensus.cc:359] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.047097 15278 raft_consensus.cc:385] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.047492 15278 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.048585 15278 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.052435 15278 ts_tablet_manager.cc:1434] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:02:04.054059 15278 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.052771 15277 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.057972 15277 ts_tablet_manager.cc:1403] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.168s	user 0.015s	sys 0.008s
W20260709 13:02:04.060595 15463 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:02:04.063477 15278 tablet_bootstrap.cc:654] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.060775 15277 raft_consensus.cc:359] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.064539 15277 raft_consensus.cc:385] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.064812 15277 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.065450 15277 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.067459 15277 ts_tablet_manager.cc:1434] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
W20260709 13:02:04.068629 15051 leader_election.cc:343] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.069137 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:04.070431 15419 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.048453 15084 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:04.073402 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:04.071290 14984 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.074453 15417 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:04.075326 15417 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:04.076737 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:04.077555 15088 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:04.076546 15417 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.081820 15279 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.082131 15308 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.082370 15279 ts_tablet_manager.cc:1403] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.181s	user 0.020s	sys 0.019s
I20260709 13:02:04.083936 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.085260 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:04.086192 15308 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.081818 14984 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.085289 15279 raft_consensus.cc:359] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.089926 15278 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.105057 15279 raft_consensus.cc:385] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.105319 15278 ts_tablet_manager.cc:1403] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.001s
I20260709 13:02:04.105418 15279 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.106258 15279 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.108347 15278 raft_consensus.cc:359] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.109654 15278 raft_consensus.cc:385] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.109969 15278 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.089468 15308 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:04.110903 15279 ts_tablet_manager.cc:1434] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:02:04.111986 15279 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:04.112514 15278 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.114941 15278 ts_tablet_manager.cc:1434] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:02:04.115943 15278 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.096535 15277 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.125051 15279 tablet_bootstrap.cc:654] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.130823 15471 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:04.127951 15277 tablet_bootstrap.cc:654] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.132251 15278 tablet_bootstrap.cc:654] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.131788 14984 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.154510 15279 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.155287 15279 ts_tablet_manager.cc:1403] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.004s
I20260709 13:02:04.164371 15279 raft_consensus.cc:359] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.165211 15279 raft_consensus.cc:385] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.165583 15279 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.166396 15279 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.167737 15278 tablet_bootstrap.cc:492] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.168234 15278 ts_tablet_manager.cc:1403] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.013s
I20260709 13:02:04.169020 15279 ts_tablet_manager.cc:1434] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:02:04.170091 15279 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:04.171341 15278 raft_consensus.cc:359] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.172117 15278 raft_consensus.cc:385] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.172420 15278 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.173518 15277 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.173991 15277 ts_tablet_manager.cc:1403] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.015s
I20260709 13:02:04.176601 15277 raft_consensus.cc:359] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.177400 15277 raft_consensus.cc:385] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.177672 15277 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.178315 15277 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.181770 15278 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.182629 15277 ts_tablet_manager.cc:1434] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:02:04.183701 15277 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.191859 15277 tablet_bootstrap.cc:654] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.192034 15278 ts_tablet_manager.cc:1434] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 13:02:04.193265 15278 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.179128 15279 tablet_bootstrap.cc:654] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.199663 15278 tablet_bootstrap.cc:654] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.203189 15277 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.204409 15277 ts_tablet_manager.cc:1403] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.003s
I20260709 13:02:04.207911 15277 raft_consensus.cc:359] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.208571 15277 raft_consensus.cc:385] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.208827 15277 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.209470 15277 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.209775 15278 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.210265 15278 ts_tablet_manager.cc:1403] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:02:04.211100 15277 ts_tablet_manager.cc:1434] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:02:04.212028 15277 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.213011 15278 raft_consensus.cc:359] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.213754 15278 raft_consensus.cc:385] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.214035 15278 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.214787 15278 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.215813 15279 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.216369 15279 ts_tablet_manager.cc:1403] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.004s
I20260709 13:02:04.217304 15460 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.218029 15277 tablet_bootstrap.cc:654] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.217772 15460 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.222635 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.223351 15237 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:04.223932 15460 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.224023 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.225169 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.228180 15278 ts_tablet_manager.cc:1434] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:02:04.229866 15460 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:02:04.230113 15054 leader_election.cc:343] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.230320 15460 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.230700 15278 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.230899 15460 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.230222 15336 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.233417 15277 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.233131 15336 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.234011 15277 ts_tablet_manager.cc:1403] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.002s
I20260709 13:02:04.237910 15278 tablet_bootstrap.cc:654] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.237504 15277 raft_consensus.cc:359] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.238336 15277 raft_consensus.cc:385] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.238713 15277 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.239567 15277 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.238617 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.241369 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:04.242266 15054 leader_election.cc:343] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.242887 15277 ts_tablet_manager.cc:1434] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:02:04.243716 15279 raft_consensus.cc:359] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:04.244210 15051 leader_election.cc:343] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.244545 15277 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.244776 15051 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:04.245903 15419 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.244562 15279 raft_consensus.cc:385] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.247531 15279 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.248284 15471 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.248258 15279 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.249042 15471 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.250447 15460 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.252440 15471 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.252807 15460 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.254070 15279 ts_tablet_manager.cc:1434] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 13:02:04.254972 15277 tablet_bootstrap.cc:654] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.257205 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.255139 15279 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:04.261819 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:04.264062 15128 leader_election.cc:343] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.265143 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.265805 15237 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:04.267055 15054 leader_election.cc:343] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.268980 15279 tablet_bootstrap.cc:654] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.269349 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.274959 15233 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:04.281327 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:04.282541 15308 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.282953 15308 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.283310 15308 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.289000 15417 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.289569 15417 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.290500 15277 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.291196 15277 ts_tablet_manager.cc:1403] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.011s
I20260709 13:02:04.292367 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.292876 15417 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:04.293145 15084 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:04.293426 15336 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.294703 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.294834 15277 raft_consensus.cc:359] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.295804 15376 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.296005 15277 raft_consensus.cc:385] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.296272 15376 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.296429 15161 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.296772 15376 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.296502 15277 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.298113 15277 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.305634 15277 ts_tablet_manager.cc:1434] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:02:04.307108 15277 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.313005 15376 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.319568 15279 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.320355 15279 ts_tablet_manager.cc:1403] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.001s
I20260709 13:02:04.322572 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:04.323310 15084 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.323665 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.324095 15279 raft_consensus.cc:359] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.325146 15279 raft_consensus.cc:385] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.325497 15279 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.329861 15419 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.330435 15419 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.332809 15419 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.334894 15334 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:02:04.337471 15279 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.338714 15435 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:04.339816 15279 ts_tablet_manager.cc:1434] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:02:04.338714 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.344830 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.345252 15435 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:04.346791 15435 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.347409 15279 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:04.343256 15162 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:04.347580 15435 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.350310 15054 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.352470 15419 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.352929 15419 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.353365 15419 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.354813 15277 tablet_bootstrap.cc:654] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:04.367877 15203 leader_election.cc:343] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.369470 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.370136 15435 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.370267 15160 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
W20260709 13:02:04.385558 15051 leader_election.cc:343] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.370939 15279 tablet_bootstrap.cc:654] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.372987 15237 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:04.375367 15277 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.389338 15277 ts_tablet_manager.cc:1403] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.003s
I20260709 13:02:04.391278 15054 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.392221 15334 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.392647 15334 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.393042 15334 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.393114 15277 raft_consensus.cc:359] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.394117 15277 raft_consensus.cc:385] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.394450 15277 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.398201 15376 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:04.399127 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.401525 15376 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.402930 15376 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.399887 15084 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:04.407684 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.408756 15458 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:04.409299 15458 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:04.410331 15458 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.412516 15435 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:04.419682 15419 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.420858 15334 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.421765 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:04.448513 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:04.449556 15162 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.459231 15435 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:04.459961 15334 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.424805 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:04.382979 15308 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.428383 15279 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.429013 15376 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:04.459360 15051 leader_election.cc:343] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.462512 15279 ts_tablet_manager.cc:1403] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.122s	user 0.016s	sys 0.000s
I20260709 13:02:04.423563 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.468891 15277 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:04.470995 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45032us
I20260709 13:02:04.478004 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.470327 15460 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.484735 15203 leader_election.cc:343] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.485340 15277 ts_tablet_manager.cc:1434] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.000s
W20260709 13:02:04.488364 15202 leader_election.cc:343] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.483349 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.485735 15460 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.494450 15308 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.499722 15162 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:04.470948 15334 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:02:04.504982 15054 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.505744 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:04.507431 15277 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.502411 15279 raft_consensus.cc:359] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.510836 15279 raft_consensus.cc:385] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.510142 15494 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:02:04.510393 15419 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.506856 15163 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.517689 15435 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:04.518798 15435 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.514063 15279 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.512336 15334 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:02:04.549371 15492 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:02:04.524878 15334 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:04.593667 15354 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.595482 15279 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.596186 15354 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.598076 15279 ts_tablet_manager.cc:1434] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.135s	user 0.024s	sys 0.049s
I20260709 13:02:04.598546 15354 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.599241 15419 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.599812 15419 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.600653 15336 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.601181 15336 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.602624 15419 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.603201 15336 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.571703 15334 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.599061 15279 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:04.621335 15460 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:04.624471 15203 leader_election.cc:343] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.625408 15203 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:04.590348 15163 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
W20260709 13:02:04.639384 15493 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.190s	user 0.000s	sys 0.005s
I20260709 13:02:04.628325 15376 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.649765 15278 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.650310 15278 ts_tablet_manager.cc:1403] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.420s	user 0.096s	sys 0.150s
I20260709 13:02:04.648180 14983 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.651973 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.663201 15376 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.672240 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.678129 15277 tablet_bootstrap.cc:654] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.675221 15376 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.676578 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.681958 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.685781 15162 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:04.688922 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.693239 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:04.695078 15051 leader_election.cc:343] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.697214 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.699378 15054 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.699860 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.700542 15237 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:04.700795 15054 leader_election.cc:343] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.702189 15054 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:04.701076 15460 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:04.693472 15279 tablet_bootstrap.cc:654] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.705561 15497 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.706051 15497 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.707442 15497 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.695106 15161 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:04.710165 15460 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:04.713477 15376 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:04.713406 15234 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:04.723194 15278 raft_consensus.cc:359] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.724198 15278 raft_consensus.cc:385] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.724560 15278 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.725711 15500 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:02:04.733784 15278 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.728962 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:04.742268 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.747432 15278 ts_tablet_manager.cc:1434] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 13:02:04.751302 15278 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.741278 15460 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:04.745968 15051 leader_election.cc:343] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:04.754091 15051 leader_election.cc:343] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:04.734745 15417 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:02:04.755298 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:04.763854 15500 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.825872 15497 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.827713 15086 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:02:04.847467 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 83168.5us
I20260709 13:02:04.851536 15277 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:04.851943 15277 ts_tablet_manager.cc:1403] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.345s	user 0.007s	sys 0.041s
W20260709 13:02:04.845448 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79553.1us
I20260709 13:02:04.859285 15277 raft_consensus.cc:359] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.860024 15277 raft_consensus.cc:385] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.860308 15277 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.861155 15277 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.847366 15237 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:04.864868 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:04.863554 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:04.865456 15083 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.865617 15159 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:04.869447 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.870350 15085 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:04.872979 15277 ts_tablet_manager.cc:1434] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:02:04.877535 15203 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:04.881290 15497 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:04.881075 15126 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.884179 15277 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:04.872565 15051 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:02:04.910697 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:04.884627 15458 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.920020 15458 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.920439 15458 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:04.895334 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:04.920140 15278 tablet_bootstrap.cc:654] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.920853 14986 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.924253 15083 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:04.925984 15161 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:04.921254 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47166.4us
I20260709 13:02:04.925714 15497 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:04.892320 15494 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
W20260709 13:02:04.927402 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46429.3us
I20260709 13:02:04.949309 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.952512 15511 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:04.953091 15511 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:04.953858 15511 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.957122 15279 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:04.957746 15279 ts_tablet_manager.cc:1403] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.359s	user 0.035s	sys 0.034s
I20260709 13:02:04.962373 15277 tablet_bootstrap.cc:654] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:04.968688 15278 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:04.969445 15278 ts_tablet_manager.cc:1403] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.012s
I20260709 13:02:04.972061 15279 raft_consensus.cc:359] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.974103 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:04.977938 15376 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.978484 15376 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.981544 15376 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:04.981776 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:04.982589 15086 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:04.972796 15279 raft_consensus.cc:385] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.984395 15279 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.988521 15278 raft_consensus.cc:359] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.990264 15278 raft_consensus.cc:385] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:04.989256 15279 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.990623 15278 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:04.991492 15278 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:04.993435 15161 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:04.993636 15278 ts_tablet_manager.cc:1434] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:02:04.993976 15308 raft_consensus.cc:2764] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:02:04.994752 15278 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:04.995062 15202 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:04.997606 15417 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:04.998065 15417 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.998384 15417 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:04.990633 15203 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.997619 15279 ts_tablet_manager.cc:1434] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:02:05.005734 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.007143 15054 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:05.015280 15458 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.016633 15234 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:05.019821 15460 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.020571 15460 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:05.021728 15460 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:05.022495 15203 leader_election.cc:343] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.037300 14983 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.038590 14983 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.045254 15279 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.050726 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:05.051906 15500 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:05.061798 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:05.062700 15158 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:05.066862 15278 tablet_bootstrap.cc:654] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.064289 15458 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.056361 15417 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.067279 14986 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.071483 15203 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.072464 15417 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.073299 15376 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.073637 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:05.074306 15083 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.075302 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:05.078301 15203 leader_election.cc:343] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.084043 15083 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:05.089651 15277 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.090222 15277 ts_tablet_manager.cc:1403] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.215s	user 0.016s	sys 0.013s
I20260709 13:02:05.091859 15279 tablet_bootstrap.cc:654] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.093170 15202 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.093216 15277 raft_consensus.cc:359] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.094173 15277 raft_consensus.cc:385] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.094481 15458 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.094591 15277 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.095175 15458 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:05.095454 15277 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.096444 15419 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.096952 15419 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.099306 15419 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.102130 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:05.104071 15054 leader_election.cc:343] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.095960 15458 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.107846 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.108551 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:05.109923 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.111006 15419 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:05.111423 15419 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.111784 15419 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.115665 15277 ts_tablet_manager.cc:1434] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:02:05.116858 15277 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.118747 15419 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.122534 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:05.122730 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:05.123247 15235 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:05.126045 15054 leader_election.cc:343] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:05.130867 15054 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.131392 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:05.133879 15419 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.136123 15279 tablet_bootstrap.cc:492] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.136618 15279 ts_tablet_manager.cc:1403] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.096s	user 0.006s	sys 0.011s
I20260709 13:02:05.139396 15279 raft_consensus.cc:359] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.140198 15279 raft_consensus.cc:385] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.140483 15279 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.141886 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.143710 15278 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:05.144716 15278 ts_tablet_manager.cc:1403] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.150s	user 0.053s	sys 0.029s
I20260709 13:02:05.145124 15500 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.145548 14983 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.147203 15500 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:05.148123 15500 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.151077 15277 tablet_bootstrap.cc:654] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.151232 15278 raft_consensus.cc:359] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.152055 15278 raft_consensus.cc:385] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.152441 15278 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.153319 15278 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.156288 15278 ts_tablet_manager.cc:1434] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:02:05.157008 15234 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.157733 15278 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:05.141211 15279 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.163962 15336 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:02:05.177383 15278 tablet_bootstrap.cc:654] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.179386 15279 ts_tablet_manager.cc:1434] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.000s
I20260709 13:02:05.180598 15279 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.184629 15354 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.185297 15354 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.186578 15511 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.187103 15279 tablet_bootstrap.cc:654] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.187644 15354 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.187249 15511 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.189617 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.190472 15088 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:05.192581 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.193617 15235 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:05.194259 15419 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:05.192636 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:05.198065 15308 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:05.198649 15308 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.199095 15308 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.210191 15511 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.212577 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.215507 15277 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.216017 15277 ts_tablet_manager.cc:1403] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.099s	user 0.007s	sys 0.011s
I20260709 13:02:05.218720 15277 raft_consensus.cc:359] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.219411 15277 raft_consensus.cc:385] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.219660 15277 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.220992 15277 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.223156 15277 ts_tablet_manager.cc:1434] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:05.224108 15277 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.231554 15277 tablet_bootstrap.cc:654] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:05.234283 15051 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.237955 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.248129 15278 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:05.248654 15278 ts_tablet_manager.cc:1403] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.091s	user 0.041s	sys 0.008s
I20260709 13:02:05.251454 15278 raft_consensus.cc:359] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.252117 15278 raft_consensus.cc:385] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.252378 15278 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.252502 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.253341 15511 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.253141 15278 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.254240 15279 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.254678 15279 ts_tablet_manager.cc:1403] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.074s	user 0.036s	sys 0.023s
I20260709 13:02:05.255565 15277 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.255978 15277 ts_tablet_manager.cc:1403] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.002s
I20260709 13:02:05.257390 15279 raft_consensus.cc:359] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.258112 15279 raft_consensus.cc:385] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.258365 15279 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.258985 15279 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.261056 15279 ts_tablet_manager.cc:1434] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:02:05.262262 15279 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.262534 15308 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:05.264688 15054 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.13.163.254:45133, user_credentials={real_user=slave}} blocked reactor thread for 84518.7us
I20260709 13:02:05.266311 15277 raft_consensus.cc:359] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.266983 15277 raft_consensus.cc:385] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.267251 15277 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.268050 15279 tablet_bootstrap.cc:654] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.267856 15277 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.269984 15277 ts_tablet_manager.cc:1434] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:02:05.271576 15277 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.276176 14986 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:05.277786 15277 tablet_bootstrap.cc:654] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.279179 15278 ts_tablet_manager.cc:1434] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.001s
I20260709 13:02:05.288589 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:05.289244 15088 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.289801 15278 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:05.279775 15308 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.281424 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:05.281921 15419 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.296819 15278 tablet_bootstrap.cc:654] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.297747 15235 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.296721 15088 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:05.305477 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:05.308864 15308 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.309615 15308 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:05.311139 15235 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:05.310690 15308 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.310853 14985 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.314723 15277 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.315263 15277 ts_tablet_manager.cc:1403] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.002s
I20260709 13:02:05.317265 15279 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.317770 15279 ts_tablet_manager.cc:1403] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.027s
I20260709 13:02:05.318113 15277 raft_consensus.cc:359] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.318894 15277 raft_consensus.cc:385] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.321133 15279 raft_consensus.cc:359] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.321899 15279 raft_consensus.cc:385] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.322325 15279 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.323021 15279 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.326462 15279 ts_tablet_manager.cc:1434] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:05.327507 15279 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.334458 15279 tablet_bootstrap.cc:654] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.338728 15278 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:05.339294 15278 ts_tablet_manager.cc:1403] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.000s
I20260709 13:02:05.342180 15278 raft_consensus.cc:359] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.342913 15278 raft_consensus.cc:385] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.343222 15278 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.343822 15277 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.344686 15277 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.347215 15277 ts_tablet_manager.cc:1434] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:02:05.348356 15277 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.349134 15278 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.351835 15278 ts_tablet_manager.cc:1434] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 13:02:05.353523 15278 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:05.355391 15277 tablet_bootstrap.cc:654] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.367873 15277 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.368369 15277 ts_tablet_manager.cc:1403] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:02:05.375651 15279 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.376192 15279 ts_tablet_manager.cc:1403] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.006s
I20260709 13:02:05.376668 15277 raft_consensus.cc:359] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.377501 15277 raft_consensus.cc:385] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.377779 15277 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.378482 15277 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.379585 15279 raft_consensus.cc:359] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.380311 15279 raft_consensus.cc:385] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.380600 15279 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.381289 15277 ts_tablet_manager.cc:1434] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:02:05.381389 15279 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.402026 15277 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.397226 14984 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.402640 15278 tablet_bootstrap.cc:654] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.414067 15277 tablet_bootstrap.cc:654] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.415645 15278 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:05.415879 15279 ts_tablet_manager.cc:1434] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 13:02:05.416224 15278 ts_tablet_manager.cc:1403] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 13:02:05.417403 15279 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.427767 15277 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.428310 15277 ts_tablet_manager.cc:1403] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.003s
I20260709 13:02:05.429270 15279 tablet_bootstrap.cc:654] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.429304 15278 raft_consensus.cc:359] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.432772 15278 raft_consensus.cc:385] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.432404 15277 raft_consensus.cc:359] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.433172 15278 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.433950 15277 raft_consensus.cc:385] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.434417 15277 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.433923 15278 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.435099 15308 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.435102 15277 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.435689 15308 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.437670 15277 ts_tablet_manager.cc:1434] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:02:05.437732 15308 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.438751 15277 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.440965 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.441685 15088 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:05.443125 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.444721 15128 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:05.446470 15277 tablet_bootstrap.cc:654] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.448271 15308 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:05.448086 15354 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:05.449061 15126 leader_election.cc:343] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.449332 15354 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.451713 15354 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.455298 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.457314 15460 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.457849 15460 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.460160 15460 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.460719 15460 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:05.463358 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.448612 15308 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.465525 15051 leader_election.cc:343] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.463460 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.466321 15308 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.471925 15159 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.467104 15279 tablet_bootstrap.cc:492] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.466925 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.473486 15279 ts_tablet_manager.cc:1403] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.008s
I20260709 13:02:05.475363 15278 ts_tablet_manager.cc:1434] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:02:05.475718 15126 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.485751 15354 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:05.481249 15054 leader_election.cc:343] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.487461 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.476253 15279 raft_consensus.cc:359] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.488914 15279 raft_consensus.cc:385] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.489398 15279 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.491350 15278 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:05.494007 15497 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.496466 15497 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.497301 15497 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.499825 15471 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.500465 15497 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.494575 15279 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.502758 15279 ts_tablet_manager.cc:1434] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:02:05.504182 15279 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:05.518985 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:05.523118 15054 leader_election.cc:343] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.494850 15308 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.528205 15308 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.531303 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:05.537750 15051 leader_election.cc:343] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.538282 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.541347 15460 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.541724 15336 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.543977 15158 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.546613 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:05.547267 15237 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.552898 15277 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.553596 15277 ts_tablet_manager.cc:1403] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.115s	user 0.036s	sys 0.037s
I20260709 13:02:05.578146 15458 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:05.615793 15417 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.616534 15417 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.634073 15279 tablet_bootstrap.cc:654] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.634177 15163 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.634859 15277 raft_consensus.cc:359] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.646165 15277 raft_consensus.cc:385] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.646476 15277 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.647118 15277 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.652077 15417 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.652469 15277 ts_tablet_manager.cc:1434] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.099s	user 0.024s	sys 0.037s
I20260709 13:02:05.652982 15237 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:05.634907 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:02:05.659862 15114 tablet.cc:2406] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:02:05.661916 15088 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.663648 15277 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:05.667299 15278 tablet_bootstrap.cc:654] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.688830 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.692953 15500 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.699087 15376 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.699817 15376 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.692997 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:05.702594 15376 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.680708 15088 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:05.706144 15308 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.706756 15471 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.706816 15308 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:05.707432 15471 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.707928 15308 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.728600 15417 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.712306 15471 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.729423 15417 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.744634 15419 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.731406 15460 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.745396 15460 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.745214 15419 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.747319 15419 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.727046 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.748919 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.754739 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.755061 15085 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:05.757042 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:05.758134 15458 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:05.758515 15458 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.758436 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.759083 15458 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.759244 15088 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:05.748493 15511 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:05.761771 15054 leader_election.cc:343] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.762068 15279 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.762601 15279 ts_tablet_manager.cc:1403] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.259s	user 0.016s	sys 0.005s
I20260709 13:02:05.763054 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:05.764598 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.765446 15376 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:05.765861 15376 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.766242 15376 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.768354 15277 tablet_bootstrap.cc:654] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.747331 15460 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:05.772512 15054 leader_election.cc:343] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.776381 15458 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.788369 15279 raft_consensus.cc:359] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.789000 15417 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.789176 15279 raft_consensus.cc:385] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.789460 15279 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.790109 15279 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.773620 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:05.778496 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 69249.1us
I20260709 13:02:05.798997 15279 ts_tablet_manager.cc:1434] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 13:02:05.803984 15163 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.805070 15551 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.798380 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.814106 15278 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:05.814775 15278 ts_tablet_manager.cc:1403] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.324s	user 0.061s	sys 0.080s
I20260709 13:02:05.817788 15203 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.805750 15551 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.818742 15458 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.820417 15551 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:05.823455 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:05.810415 15554 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:02:05.826820 15554 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.830816 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:05.830894 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:05.832562 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:05.836470 15051 leader_election.cc:343] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.837041 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.837934 15500 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:05.839434 15552 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:05.837798 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:05.841707 15203 leader_election.cc:343] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.844548 15278 raft_consensus.cc:359] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.845331 15278 raft_consensus.cc:385] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.845629 15279 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
W20260709 13:02:05.846669 15051 leader_election.cc:343] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.847215 15051 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.845642 15278 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.848628 15278 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.842465 15376 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.839493 15237 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.856014 15278 ts_tablet_manager.cc:1434] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:02:05.859092 15376 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:05.853807 15497 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.863976 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:05.864753 14983 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:05.864995 15203 leader_election.cc:343] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.867002 15308 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.865602 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:05.860671 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50588.6us
I20260709 13:02:05.870599 15128 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.871059 15159 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.879395 15278 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:05.878981 15354 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:05.886591 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:05.886897 15279 tablet_bootstrap.cc:654] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:05.886991 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:05.887282 15086 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.887588 15088 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:05.879187 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:05.881951 15336 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:02:05.907881 15277 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:05.909231 15277 ts_tablet_manager.cc:1403] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.246s	user 0.006s	sys 0.018s
I20260709 13:02:05.917433 15088 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
W20260709 13:02:05.920084 15203 leader_election.cc:343] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:05.927516 15159 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:05.921521 15086 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:05.932466 15203 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:05.934301 15417 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.934806 15417 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:05.935273 15128 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:05.935700 15417 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.937371 15354 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:05.943365 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:05.952312 15308 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.949882 15277 raft_consensus.cc:359] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.953799 15277 raft_consensus.cc:385] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.954144 15277 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.955080 15277 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.960767 15277 ts_tablet_manager.cc:1434] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:02:05.961469 15556 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:05.963587 15556 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:05.965679 15556 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.980942 15279 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:05.981467 15279 ts_tablet_manager.cc:1403] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.136s	user 0.009s	sys 0.009s
I20260709 13:02:05.984200 15279 raft_consensus.cc:359] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.985411 15279 raft_consensus.cc:385] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:05.985788 15279 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:05.986454 15279 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:05.988582 15279 ts_tablet_manager.cc:1434] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:02:05.988714 15278 tablet_bootstrap.cc:654] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.007848 15279 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.010923 15278 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:06.014016 15277 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.024302 15277 tablet_bootstrap.cc:654] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.033757 15278 ts_tablet_manager.cc:1403] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.004s
I20260709 13:02:06.051705 15500 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.052452 15500 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.047549 15278 raft_consensus.cc:359] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.053872 15278 raft_consensus.cc:385] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.054247 15278 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.054952 15278 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.056679 15500 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.057281 15278 ts_tablet_manager.cc:1434] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:02:06.061614 15278 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:06.062004 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:06.064002 15054 leader_election.cc:343] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:06.065531 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:06.061880 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:06.070520 15051 leader_election.cc:343] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.071004 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:06.071698 15336 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:06.076414 15278 tablet_bootstrap.cc:654] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.077447 15279 tablet_bootstrap.cc:654] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.084060 15277 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.084537 15277 ts_tablet_manager.cc:1403] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.003s
I20260709 13:02:06.087239 15277 raft_consensus.cc:359] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.087920 15277 raft_consensus.cc:385] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.088172 15277 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.089251 15277 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.091399 15277 ts_tablet_manager.cc:1434] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:02:06.092406 15277 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.097843 15277 tablet_bootstrap.cc:654] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.101827 15354 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.102411 15354 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.105510 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.106314 15086 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:06.107898 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:06.108997 15308 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:06.109395 15308 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.109746 15308 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.114785 15354 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.127656 15279 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.128314 15277 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.125614 15278 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:06.134646 15277 ts_tablet_manager.cc:1403] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.042s	user 0.030s	sys 0.011s
I20260709 13:02:06.135308 15279 ts_tablet_manager.cc:1403] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.128s	user 0.017s	sys 0.015s
I20260709 13:02:06.134734 15278 ts_tablet_manager.cc:1403] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.012s
I20260709 13:02:06.129866 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.137519 15237 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:06.137669 15277 raft_consensus.cc:359] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.138345 15277 raft_consensus.cc:385] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.138607 15277 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.139204 15277 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.139339 15278 raft_consensus.cc:359] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.140045 15278 raft_consensus.cc:385] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.140326 15278 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.141247 15277 ts_tablet_manager.cc:1434] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:02:06.120739 15308 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.142208 15277 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.142678 15279 raft_consensus.cc:359] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.143323 15279 raft_consensus.cc:385] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.143600 15279 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.144373 15279 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.146740 15279 ts_tablet_manager.cc:1434] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:02:06.147732 15279 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.148008 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:06.148644 15277 tablet_bootstrap.cc:654] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.148608 15086 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.149649 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:06.150409 15237 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.153869 15279 tablet_bootstrap.cc:654] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.156826 15237 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:06.146859 15308 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.157982 15126 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:06.155699 15086 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:06.159026 15308 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:06.159475 15308 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:06.160221 15308 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.163013 15277 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.163434 15277 ts_tablet_manager.cc:1403] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.003s
I20260709 13:02:06.149789 14984 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:06.165797 14984 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:06.166599 15277 raft_consensus.cc:359] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.167331 15277 raft_consensus.cc:385] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.167595 15277 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.168303 15279 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.168784 15279 ts_tablet_manager.cc:1403] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:02:06.168422 15277 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.171070 15277 ts_tablet_manager.cc:1434] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:02:06.171527 15279 raft_consensus.cc:359] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.172297 15279 raft_consensus.cc:385] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.172399 15277 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.172504 15279 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.173374 15279 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.195991 15279 ts_tablet_manager.cc:1434] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:02:06.197325 15279 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.351938 15554 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.352746 15554 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.356081 15554 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:06.348590 15277 tablet_bootstrap.cc:654] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.396764 15511 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:06.141000 15278 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.399569 15354 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.400099 15354 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.402837 15278 ts_tablet_manager.cc:1434] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.266s	user 0.005s	sys 0.000s
I20260709 13:02:06.403863 15278 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:06.406023 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.410518 15278 tablet_bootstrap.cc:654] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.427649 15278 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:06.428157 15278 ts_tablet_manager.cc:1403] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.008s
I20260709 13:02:06.431304 15278 raft_consensus.cc:359] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.432149 15278 raft_consensus.cc:385] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.432466 15278 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.433506 15278 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.436468 15278 ts_tablet_manager.cc:1434] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:02:06.438081 15278 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:06.446316 15278 tablet_bootstrap.cc:654] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:06.452080 15203 leader_election.cc:343] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.453966 15555 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:06.458330 15279 tablet_bootstrap.cc:654] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.472426 15354 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.481853 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.487993 15279 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.488548 15279 ts_tablet_manager.cc:1403] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.291s	user 0.008s	sys 0.002s
I20260709 13:02:06.496819 15278 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:06.503533 15278 ts_tablet_manager.cc:1403] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.013s
I20260709 13:02:06.506410 15277 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.506933 15277 ts_tablet_manager.cc:1403] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.335s	user 0.033s	sys 0.065s
I20260709 13:02:06.509816 15277 raft_consensus.cc:359] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.510565 15277 raft_consensus.cc:385] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.510896 15277 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.511538 15277 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.500325 15279 raft_consensus.cc:359] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.513728 15279 raft_consensus.cc:385] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.513998 15279 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.514676 15279 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.516916 15279 ts_tablet_manager.cc:1434] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:02:06.517887 15279 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.518618 15278 raft_consensus.cc:359] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.524197 15279 tablet_bootstrap.cc:654] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.519317 15278 raft_consensus.cc:385] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.525068 15278 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.528147 15277 ts_tablet_manager.cc:1434] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:02:06.529744 15277 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.525895 15278 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.532905 15500 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.533655 15500 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.534376 15278 ts_tablet_manager.cc:1434] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:02:06.535499 15278 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:06.535766 15500 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.560202 15590 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.560282 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.561338 15086 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:06.561331 15590 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.564138 15590 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.589561 15590 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.590174 15590 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.592975 15590 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.595927 15590 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.596541 15590 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.599515 15590 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.620368 15583 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:06.623559 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.624392 15088 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:06.630782 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:06.634123 15054 leader_election.cc:343] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.637652 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:06.638840 15054 leader_election.cc:343] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.645467 15128 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:06.647203 15583 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.653461 15277 tablet_bootstrap.cc:654] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.656569 15583 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:06.661759 15277 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.662504 15277 ts_tablet_manager.cc:1403] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.133s	user 0.012s	sys 0.001s
I20260709 13:02:06.663290 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:06.665299 15054 leader_election.cc:343] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.667273 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.666757 15277 raft_consensus.cc:359] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.667850 15162 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:06.667891 15277 raft_consensus.cc:385] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.668382 15277 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.668769 15054 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:06.669591 15334 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:06.669391 15277 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.670019 15334 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.670416 15334 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.671309 15277 ts_tablet_manager.cc:1434] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:02:06.672667 15277 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.682685 15277 tablet_bootstrap.cc:654] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.682334 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:06.692463 15555 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.693027 15555 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.695093 15555 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:06.696157 15277 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.696914 15277 ts_tablet_manager.cc:1403] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:02:06.700922 15277 raft_consensus.cc:359] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.713106 15277 raft_consensus.cc:385] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.713640 15277 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.704453 15354 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:06.706354 15278 tablet_bootstrap.cc:654] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.716042 15277 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.718655 15277 ts_tablet_manager.cc:1434] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:02:06.720293 15277 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.722167 15336 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.722913 15336 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.727222 15277 tablet_bootstrap.cc:654] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.730599 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.731442 15087 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:06.734293 15336 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.741503 15277 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.742261 15277 ts_tablet_manager.cc:1403] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.011s
I20260709 13:02:06.745293 15592 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.745504 15277 raft_consensus.cc:359] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.745899 15592 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.746299 15277 raft_consensus.cc:385] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.746740 15277 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.747486 15277 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.748260 15592 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:06.751130 15277 ts_tablet_manager.cc:1434] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:02:06.752569 15277 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.764493 15277 tablet_bootstrap.cc:654] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.769264 15554 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.769907 15554 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.772420 15554 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:06.800266 15277 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.800781 15277 ts_tablet_manager.cc:1403] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.008s
I20260709 13:02:06.803449 15277 raft_consensus.cc:359] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.804289 15277 raft_consensus.cc:385] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.804618 15277 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.805361 15277 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.807693 15277 ts_tablet_manager.cc:1434] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:02:06.811087 15279 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.812137 15279 ts_tablet_manager.cc:1403] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.294s	user 0.012s	sys 0.014s
I20260709 13:02:06.814998 15279 raft_consensus.cc:359] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.815769 15279 raft_consensus.cc:385] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.816048 15279 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.816761 15279 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.817896 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.818540 15237 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:06.819083 15279 ts_tablet_manager.cc:1434] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:02:06.820101 15279 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.820180 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.808727 15277 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
W20260709 13:02:06.820719 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 97931.4us
I20260709 13:02:06.821516 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:06.823067 15554 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:06.823477 15554 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.823882 15554 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.679384 15334 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.827468 15277 tablet_bootstrap.cc:654] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.829962 15568 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:06.820241 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.831434 15237 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:06.838029 15279 tablet_bootstrap.cc:654] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.842540 15334 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.843750 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:06.844555 15163 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.855330 15277 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.855841 15277 ts_tablet_manager.cc:1403] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.001s
I20260709 13:02:06.856240 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.857868 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.858621 15277 raft_consensus.cc:359] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.859386 15277 raft_consensus.cc:385] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.859655 15277 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
W20260709 13:02:06.861326 15054 leader_election.cc:343] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.862823 15279 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.862802 15554 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.863471 15279 ts_tablet_manager.cc:1403] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.005s
I20260709 13:02:06.866103 15279 raft_consensus.cc:359] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.866781 15279 raft_consensus.cc:385] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.866552 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.867059 15279 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.867769 15279 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.866542 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:06.869895 15279 ts_tablet_manager.cc:1434] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:02:06.870844 15279 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.870759 15598 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:06.871206 15598 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.871585 15598 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.872144 15278 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:06.872649 15278 ts_tablet_manager.cc:1403] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.337s	user 0.076s	sys 0.074s
I20260709 13:02:06.873085 15277 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.875312 15277 ts_tablet_manager.cc:1434] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:02:06.876324 15277 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.883594 15278 raft_consensus.cc:359] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.884584 15278 raft_consensus.cc:385] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.884958 15278 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.885780 15500 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.886296 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.885748 15278 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.887188 15082 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:06.887907 15279 tablet_bootstrap.cc:654] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.889637 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.893792 15082 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:06.889436 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:06.892906 15277 tablet_bootstrap.cc:654] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:06.886327 15500 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.899775 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.900275 15279 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:06.900058 15500 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.900739 15279 ts_tablet_manager.cc:1403] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.001s
W20260709 13:02:06.901191 15054 leader_election.cc:343] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.904300 15279 raft_consensus.cc:359] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.904996 15279 raft_consensus.cc:385] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.905296 15279 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.905849 15279 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.906258 15500 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.906826 15500 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.910794 15279 ts_tablet_manager.cc:1434] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:02:06.911401 15278 ts_tablet_manager.cc:1434] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.002s
I20260709 13:02:06.901091 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:06.913272 15234 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.914430 15277 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:06.914801 15278 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:06.917057 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.912025 15163 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:06.914891 15277 ts_tablet_manager.cc:1403] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.009s
I20260709 13:02:06.918445 15354 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.918498 15054 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:06.919013 15354 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:06.920248 15054 leader_election.cc:343] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:06.920889 15334 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:06.920737 15277 raft_consensus.cc:359] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.921458 15277 raft_consensus.cc:385] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:06.921504 15334 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:06.921806 15277 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:06.922187 15354 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.922535 15277 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.922567 15334 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:06.916193 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.923136 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:06.926488 15155 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:06.926345 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:06.927747 15128 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:06.924012 15500 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:06.911811 15279 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:06.924897 15277 ts_tablet_manager.cc:1434] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:02:06.929804 15277 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:06.930457 15584 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:06.930860 15584 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.931200 15584 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.939813 15051 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:06.941100 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:06.942951 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:06.943568 15086 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:06.950724 15202 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:06.935066 15554 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:06.995919 15599 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:02:06.994037 15460 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.008973 15460 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:07.025214 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 85496.7us
W20260709 13:02:07.025578 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74970us
I20260709 13:02:07.032516 15234 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:07.026235 15460 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.027225 15599 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.033275 15599 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.033609 15599 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:07.035212 15051 leader_election.cc:343] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.035682 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.036325 15500 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:07.037397 15051 leader_election.cc:343] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.038388 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:07.037792 15051 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.039498 15500 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.048434 15460 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.049045 15604 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:02:07.046782 15203 leader_election.cc:343] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.029814 15590 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.065848 15279 tablet_bootstrap.cc:654] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.070492 15419 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.077759 15419 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.080368 15419 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:07.086665 15051 leader_election.cc:343] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.087355 15051 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.095573 15277 tablet_bootstrap.cc:654] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.072912 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:07.097577 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45548.5us
W20260709 13:02:07.100751 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44161.3us
I20260709 13:02:07.101500 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.104169 15336 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:07.108882 15203 leader_election.cc:343] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.070431 15590 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.101270 15460 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.115654 15611 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.118595 15584 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.118999 15611 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.119902 15160 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:07.109642 15554 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.122002 15584 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.110987 15598 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.137326 15598 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.137753 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:07.158948 15611 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.167572 15278 tablet_bootstrap.cc:654] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.171067 15592 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.171224 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.171672 15592 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.172322 15590 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.173880 15086 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.174269 15592 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.175724 15230 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.178668 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.178951 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:07.180186 15054 leader_election.cc:343] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.183409 15279 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:07.183962 15279 ts_tablet_manager.cc:1403] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.272s	user 0.018s	sys 0.017s
W20260709 13:02:07.184926 15054 leader_election.cc:343] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:07.189694 15054 leader_election.cc:343] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:07.190639 15054 leader_election.cc:343] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.195773 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.183548 15599 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.196605 15087 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:07.203131 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.209888 15598 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.198658 15604 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.218014 15599 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.230816 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.229889 15598 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:07.234694 15568 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.233871 15598 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.235009 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.235863 15087 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:07.236043 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.235162 15568 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.237540 15568 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:07.241752 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 95115.2us
I20260709 13:02:07.243184 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.244999 15584 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.245221 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.245503 15584 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.236680 15084 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.232009 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.251396 15277 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:07.253095 15277 ts_tablet_manager.cc:1403] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.324s	user 0.007s	sys 0.029s
I20260709 13:02:07.240180 15604 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.262935 15279 raft_consensus.cc:359] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.263865 15279 raft_consensus.cc:385] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.264243 15279 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.264755 15279 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.254319 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:07.272389 15279 ts_tablet_manager.cc:1434] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.000s
I20260709 13:02:07.272799 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.289953 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.284688 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:07.293706 15229 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.294068 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.290647 15085 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:07.302769 14985 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:07.306643 15084 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.313301 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.317692 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.318835 15617 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.319325 15617 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:02:07.317157 15051 leader_election.cc:343] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.321244 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.321977 15500 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.320143 15617 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.328070 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:07.329509 15051 leader_election.cc:343] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.329845 15051 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.332032 15419 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.332485 15584 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.334997 15051 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.335736 15419 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.336153 15278 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:07.336598 15278 ts_tablet_manager.cc:1403] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.424s	user 0.009s	sys 0.029s
W20260709 13:02:07.336826 15051 leader_election.cc:343] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.335263 15277 raft_consensus.cc:359] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.337242 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.338544 15277 raft_consensus.cc:385] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.338924 15277 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
W20260709 13:02:07.340045 15203 leader_election.cc:343] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:07.342082 15203 leader_election.cc:343] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.342448 15203 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.322474 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.316965 15604 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.347247 15277 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.322964 15599 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.306844 15568 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.356074 15616 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.342687 15419 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:07.359133 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:07.359385 15554 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.363749 15620 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 13:02:07.375613 15278 raft_consensus.cc:359] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.376361 15279 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:07.376367 15278 raft_consensus.cc:385] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.338632 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.363636 15554 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.375655 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.379905 15278 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.366640 15084 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:07.387629 15278 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.389340 15128 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:07.362046 15599 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.394467 15277 ts_tablet_manager.cc:1434] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.141s	user 0.007s	sys 0.001s
I20260709 13:02:07.405967 15278 ts_tablet_manager.cc:1434] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.002s
I20260709 13:02:07.398643 15551 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.411866 15551 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.412102 15233 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:07.412261 15551 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.398221 15599 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.419644 15568 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.401279 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.424438 15278 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:07.413594 15419 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.426047 15051 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.427873 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:07.429992 15051 leader_election.cc:343] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.430451 15051 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.422168 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.433147 15334 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.433768 15334 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.434087 15334 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:07.437359 15203 leader_election.cc:343] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.440021 15277 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:07.427363 15590 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:02:07.443289 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:07.442008 15460 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.442128 15083 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.454874 15128 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.455247 15279 tablet_bootstrap.cc:654] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.458474 15622 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:02:07.463728 15203 leader_election.cc:343] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.459748 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.431595 15419 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.488629 15620 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.499231 15084 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:07.505882 15628 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:02:07.492733 15620 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.528264 15554 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.542804 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.525501 15590 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:02:07.554632 15278 tablet_bootstrap.cc:654] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:07.557211 15054 leader_election.cc:343] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.555948 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.560166 15334 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.561915 15084 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.562942 15334 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:07.567497 15203 leader_election.cc:343] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.570215 15620 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.573030 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.573792 15083 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:07.574090 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:07.574654 15109 debug-util.cc:398] Leaking SignalData structure 0x7b08001b2740 after lost signal to thread 13970
I20260709 13:02:07.574966 15232 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:02:07.575675 15109 debug-util.cc:398] Leaking SignalData structure 0x7b08000a4680 after lost signal to thread 15020
I20260709 13:02:07.573138 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:07.581238 15277 tablet_bootstrap.cc:654] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.582615 14986 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.563381 15551 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:07.578495 15109 debug-util.cc:398] Leaking SignalData structure 0x7b08001a10a0 after lost signal to thread 15112
I20260709 13:02:07.589989 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:07.593008 15054 leader_election.cc:343] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.586907 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:07.597299 15419 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.597761 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.604710 15235 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.608801 15202 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.609256 15279 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:07.610014 15279 ts_tablet_manager.cc:1403] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.234s	user 0.005s	sys 0.013s
W20260709 13:02:07.605829 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080020e1a0 after lost signal to thread 15187
I20260709 13:02:07.628204 15279 raft_consensus.cc:359] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.629091 15279 raft_consensus.cc:385] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.629396 15279 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.622581 15157 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:07.631855 15631 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.632405 15631 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:07.635712 15279 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.637441 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.638275 15087 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:07.640002 15631 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:07.642120 15051 leader_election.cc:343] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.647236 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.648815 15618 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:07.649746 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080028c1e0 after lost signal to thread 15261
I20260709 13:02:07.652480 15279 ts_tablet_manager.cc:1434] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.001s
I20260709 13:02:07.620330 15617 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.612210 15599 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.660445 15279 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:07.657351 15617 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.666963 15617 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.667596 15235 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:07.667846 15234 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.665087 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.673435 15624 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.674293 15051 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.678931 15419 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.679490 15419 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:07.683070 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:07.684235 15233 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.688007 15599 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.688208 15624 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.672919 15551 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.688964 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.689601 15086 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.693219 15419 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.697367 15086 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.698506 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.703668 15613 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.704339 15613 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:07.705427 15278 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:07.705914 15278 ts_tablet_manager.cc:1403] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.285s	user 0.018s	sys 0.036s
I20260709 13:02:07.701438 15599 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.705264 15613 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.714829 15599 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.713462 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:07.712800 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.719942 15083 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:07.720820 15599 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.724295 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:07.726408 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:07.727159 15087 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:07.727612 15157 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:07.728621 15202 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.730971 15554 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:07.731469 15554 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.731846 15554 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.726477 15278 raft_consensus.cc:359] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.733566 15278 raft_consensus.cc:385] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.733887 15278 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.734604 15278 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.738291 15278 ts_tablet_manager.cc:1434] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
W20260709 13:02:07.739439 15203 leader_election.cc:343] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.757747 15278 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:07.763047 15128 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:07.765237 15636 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:07.767040 14984 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:07.769474 15277 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:07.769989 15277 ts_tablet_manager.cc:1403] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.346s	user 0.016s	sys 0.014s
I20260709 13:02:07.772046 15277 raft_consensus.cc:359] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.772643 15277 raft_consensus.cc:385] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.775799 15277 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.777459 15277 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.783218 15279 tablet_bootstrap.cc:654] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.784104 15554 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:07.787317 15644 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:02:07.787569 15277 ts_tablet_manager.cc:1434] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:02:07.788897 15554 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.789093 15617 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.789938 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.790567 15083 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.788708 15277 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:07.792181 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:07.794991 15203 leader_election.cc:343] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.791765 15613 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:02:07.798199 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:07.799947 15085 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:07.799265 15617 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:07.816210 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:07.820199 15278 tablet_bootstrap.cc:654] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.820336 15085 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.821743 15202 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:07.822702 15617 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.823576 15617 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:07.825585 15617 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:07.833863 15203 leader_election.cc:343] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.837364 15277 tablet_bootstrap.cc:654] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.845252 15554 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:07.841657 14985 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.849785 15083 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:07.851085 15613 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:07.851656 15279 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:07.848898 14985 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.851138 15202 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:07.852312 15279 ts_tablet_manager.cc:1403] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.193s	user 0.016s	sys 0.003s
I20260709 13:02:07.853138 15554 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:07.854169 15554 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:07.853314 14985 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.855046 15554 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.855360 15279 raft_consensus.cc:359] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.856266 15279 raft_consensus.cc:385] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.866034 15278 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:07.866647 15278 ts_tablet_manager.cc:1403] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.001s
I20260709 13:02:07.856591 15279 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.869549 15279 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.871456 15279 ts_tablet_manager.cc:1434] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:02:07.872463 15279 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:07.883198 15278 raft_consensus.cc:359] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.884266 15278 raft_consensus.cc:385] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.885088 15279 tablet_bootstrap.cc:654] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.884651 15278 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.917523 15277 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:07.918227 15277 ts_tablet_manager.cc:1403] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.002s
I20260709 13:02:07.917626 15278 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.925254 15278 ts_tablet_manager.cc:1434] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.004s
I20260709 13:02:07.936416 15278 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:07.947360 15277 raft_consensus.cc:359] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.948096 15277 raft_consensus.cc:385] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.948349 15277 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.949182 15277 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.952405 15460 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.952894 15460 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.954875 15460 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:07.955914 15279 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:07.956525 15279 ts_tablet_manager.cc:1403] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.084s	user 0.017s	sys 0.011s
I20260709 13:02:07.960209 15278 tablet_bootstrap.cc:654] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.955969 15277 ts_tablet_manager.cc:1434] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.003s
I20260709 13:02:07.969980 15277 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
W20260709 13:02:07.961745 15203 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:07.973258 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45110.1us
I20260709 13:02:07.975256 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:07.979547 15051 leader_election.cc:343] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:07.962479 15279 raft_consensus.cc:359] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.980993 15279 raft_consensus.cc:385] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:07.981292 15279 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:07.981846 14986 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.984107 14986 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:07.987973 15277 tablet_bootstrap.cc:654] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:07.981992 15279 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:07.991421 15279 ts_tablet_manager.cc:1434] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 13:02:07.997748 15279 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:07.961295 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.004657 15279 tablet_bootstrap.cc:654] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.028071 15278 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.070132 15278 ts_tablet_manager.cc:1403] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.134s	user 0.029s	sys 0.054s
W20260709 13:02:08.074271 15054 leader_election.cc:343] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.074760 15054 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.076094 15460 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.046940 15555 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.074620 15278 raft_consensus.cc:359] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.083381 15555 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.068904 15277 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:08.084738 15277 ts_tablet_manager.cc:1403] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.000s
I20260709 13:02:08.083604 15278 raft_consensus.cc:385] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.085471 15278 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.086325 15278 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.088932 15554 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.089530 15554 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.089952 15555 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.091102 15278 ts_tablet_manager.cc:1434] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:02:08.093715 15278 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.096036 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.097195 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.099426 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:08.099707 15203 leader_election.cc:343] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.100214 15086 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:08.100391 15278 tablet_bootstrap.cc:654] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:08.100459 15203 leader_election.cc:343] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.107056 15611 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:08.110157 15202 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:08.111078 15599 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.111254 15551 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.111505 15599 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.111846 15599 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.111766 15551 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.113113 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.113852 15083 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:08.113893 15551 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.115648 15202 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:08.091976 15554 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:08.118675 15054 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:08.119167 15599 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.121196 15555 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.121618 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.121634 15555 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.123044 15555 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.126837 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.127751 15083 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:08.129263 15128 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:02:08.132740 15126 leader_election.cc:343] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.134195 15551 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.134656 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:08.134724 15551 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.135452 15551 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.143339 15551 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.131902 15555 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.145080 15083 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.147482 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:08.147887 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:08.148173 15086 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:08.149484 15203 leader_election.cc:343] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.150223 15555 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.163357 15277 raft_consensus.cc:359] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.164189 15277 raft_consensus.cc:385] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.164516 15277 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.165270 15277 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.165604 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:08.167138 15203 leader_election.cc:343] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.194610 15599 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.197583 15086 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:08.199822 15551 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.199324 15202 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:08.201678 15617 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:08.202173 15617 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:08.203226 15617 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.205390 15277 ts_tablet_manager.cc:1434] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.120s	user 0.009s	sys 0.005s
I20260709 13:02:08.206840 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:08.207520 15085 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.208679 15277 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:08.208165 15083 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:08.212397 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:08.218489 15279 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:08.219063 15279 ts_tablet_manager.cc:1403] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.227s	user 0.005s	sys 0.008s
W20260709 13:02:08.223209 15126 leader_election.cc:343] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:08.224174 15203 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:08.225164 15279 raft_consensus.cc:359] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.226869 15279 raft_consensus.cc:385] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.226157 15555 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:02:08.228163 15279 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.228945 15279 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.237046 15202 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:08.241248 15460 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.241611 15599 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:08.241766 15460 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.242333 15599 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:08.243948 15460 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.243427 15599 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.239100 15554 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:02:08.257522 15278 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.258152 15278 ts_tablet_manager.cc:1403] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.165s	user 0.009s	sys 0.008s
I20260709 13:02:08.266988 15278 raft_consensus.cc:359] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.268044 15278 raft_consensus.cc:385] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.268445 15278 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.269429 15278 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.276335 15277 tablet_bootstrap.cc:654] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.283003 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:08.298947 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 72650.2us
I20260709 13:02:08.285960 15554 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.286995 15085 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:08.293483 15279 ts_tablet_manager.cc:1434] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.001s
I20260709 13:02:08.298702 15551 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.308399 15551 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.310645 15551 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.312071 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.314631 15161 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.316272 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:08.318220 15054 leader_election.cc:343] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.324582 15278 ts_tablet_manager.cc:1434] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:02:08.326781 15624 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.327386 15624 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.328708 15278 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.331326 15624 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.332778 15554 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.333436 15128 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.334539 15551 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:08.335072 15551 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:08.336346 15279 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:08.337113 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.339524 15551 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:08.354668 15051 leader_election.cc:343] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.356873 15051 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.352216 14984 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:08.359989 14984 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:08.379128 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.380414 15083 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:08.364130 15460 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.384773 15278 tablet_bootstrap.cc:654] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.386044 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.406997 15279 tablet_bootstrap.cc:654] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.421089 15277 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:08.401400 15128 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.421794 15277 ts_tablet_manager.cc:1403] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.214s	user 0.030s	sys 0.022s
I20260709 13:02:08.426554 15624 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.410003 15087 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:08.429489 15277 raft_consensus.cc:359] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.434156 15277 raft_consensus.cc:385] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.434695 15277 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.435937 15277 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.441004 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.441825 15277 ts_tablet_manager.cc:1434] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:02:08.441892 15551 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.443616 15551 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:08.444051 15551 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.444691 15277 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:08.452554 15624 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.463644 15551 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.465883 15551 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.466725 15163 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.470043 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:02:08.470484 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:08.470657 15086 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.473248 15592 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.473449 14986 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.481366 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:08.482471 15554 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.483682 15554 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.484359 15554 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:08.485246 15554 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.488379 15554 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.490757 15592 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.494053 15278 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.494573 15278 ts_tablet_manager.cc:1403] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.166s	user 0.015s	sys 0.021s
I20260709 13:02:08.502763 15554 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:08.504626 15086 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:08.505465 15087 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:02:08.505847 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.507694 15636 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:08.508318 15636 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:08.509156 15636 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.510267 15277 tablet_bootstrap.cc:654] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.502823 15278 raft_consensus.cc:359] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.518414 15278 raft_consensus.cc:385] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.518759 15278 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.523586 15278 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.532336 15278 ts_tablet_manager.cc:1434] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.001s
I20260709 13:02:08.541038 15277 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:08.541512 15277 ts_tablet_manager.cc:1403] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.000s
I20260709 13:02:08.543267 15279 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:08.544090 15278 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.544353 15279 ts_tablet_manager.cc:1403] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.212s	user 0.011s	sys 0.009s
I20260709 13:02:08.548098 15279 raft_consensus.cc:359] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.548885 15279 raft_consensus.cc:385] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.549204 15279 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.550325 15279 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.551366 15277 raft_consensus.cc:359] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.551771 15279 ts_tablet_manager.cc:1434] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:02:08.551924 15277 raft_consensus.cc:385] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.552232 15277 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.552558 15279 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:08.552829 15277 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.560376 15278 tablet_bootstrap.cc:654] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.555191 14984 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.563799 15277 ts_tablet_manager.cc:1434] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:02:08.569278 15277 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Bootstrap starting.
I20260709 13:02:08.587093 15277 tablet_bootstrap.cc:654] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.589046 15279 tablet_bootstrap.cc:654] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.598423 15278 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.599020 15278 ts_tablet_manager.cc:1403] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.004s
I20260709 13:02:08.603981 15278 raft_consensus.cc:359] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.604700 15278 raft_consensus.cc:385] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.605005 15278 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.605783 15278 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.607432 15279 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:08.607707 15277 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: No bootstrap required, opened a new log
I20260709 13:02:08.607987 15278 ts_tablet_manager.cc:1434] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:02:08.608531 15277 ts_tablet_manager.cc:1403] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.004s
I20260709 13:02:08.609553 15278 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.610989 15618 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.608954 15279 ts_tablet_manager.cc:1403] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.005s
I20260709 13:02:08.611555 15618 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.613888 15618 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.614573 15277 raft_consensus.cc:359] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.615798 15277 raft_consensus.cc:385] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.616142 15277 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.616534 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.614056 15279 raft_consensus.cc:359] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.615278 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:08.618358 15051 leader_election.cc:343] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:08.618853 15054 leader_election.cc:343] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.619299 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.620675 15618 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.618539 15279 raft_consensus.cc:385] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.621747 15279 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.622428 15277 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.623900 15278 tablet_bootstrap.cc:654] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.623762 15279 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.632362 15277 ts_tablet_manager.cc:1434] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 13:02:08.632900 15279 ts_tablet_manager.cc:1434] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:02:08.635119 15279 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:08.642033 15279 tablet_bootstrap.cc:654] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.642532 15278 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.643036 15278 ts_tablet_manager.cc:1403] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.004s
I20260709 13:02:08.645884 15278 raft_consensus.cc:359] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.646768 15278 raft_consensus.cc:385] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.647046 15278 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.647805 15278 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.657003 15279 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:08.657825 15279 ts_tablet_manager.cc:1403] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.000s
I20260709 13:02:08.658943 15618 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.659432 15618 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.661481 15618 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.661698 15555 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:08.663476 15278 ts_tablet_manager.cc:1434] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:02:08.664534 15278 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.664439 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.667264 15279 raft_consensus.cc:359] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.668164 15279 raft_consensus.cc:385] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:02:08.668905 15051 leader_election.cc:343] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.668032 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.668635 15279 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.670292 15279 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:08.672191 15054 leader_election.cc:343] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.672627 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.673162 15279 ts_tablet_manager.cc:1434] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:08.673329 15460 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.674254 15279 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:08.681712 15551 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.682272 15551 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.684001 15279 tablet_bootstrap.cc:654] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.691174 15278 tablet_bootstrap.cc:654] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.692768 15460 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.694213 15460 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.696229 15460 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.697439 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.699657 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:08.701159 15203 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:08.702566 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.704052 15279 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:08.704620 15279 ts_tablet_manager.cc:1403] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.008s
I20260709 13:02:08.705466 15161 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:08.705617 15051 leader_election.cc:343] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.707654 15279 raft_consensus.cc:359] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.708575 15279 raft_consensus.cc:385] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.708921 15279 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.709632 15279 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.712160 15279 ts_tablet_manager.cc:1434] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:02:08.713244 15279 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
W20260709 13:02:08.715087 15054 leader_election.cc:343] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.715565 15054 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.716302 15618 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:08.719496 15279 tablet_bootstrap.cc:654] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.729308 15551 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.732913 15617 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.733712 15617 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.740252 15617 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.742137 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.743189 15086 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
W20260709 13:02:08.742077 15203 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:08.746286 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.746511 15202 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:08.747838 15599 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.748251 15599 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.748622 15599 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.749351 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.751557 15555 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.752270 15555 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.753734 15592 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.754240 15592 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.759442 15163 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.762346 15126 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:08.764353 15460 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.764919 15460 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.765836 15592 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.766956 15460 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:08.769605 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.770859 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.771639 15086 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:08.773694 15279 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
W20260709 13:02:08.773797 15054 leader_election.cc:343] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.774217 15279 ts_tablet_manager.cc:1403] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.000s
I20260709 13:02:08.777123 15599 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.780575 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:08.781499 15592 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.781927 15592 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.782279 15592 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.787890 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:08.788452 15086 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:08.788976 15203 leader_election.cc:343] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:08.790339 15279 raft_consensus.cc:359] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.791352 15279 raft_consensus.cc:385] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.791774 15279 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.793293 15617 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.793798 15617 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.792820 15279 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.786829 15555 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.816933 15278 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:08.817471 15278 ts_tablet_manager.cc:1403] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.153s	user 0.045s	sys 0.047s
I20260709 13:02:08.819157 15279 ts_tablet_manager.cc:1434] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 13:02:08.828200 15086 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:08.831862 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.832593 15278 raft_consensus.cc:359] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.846673 15278 raft_consensus.cc:385] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.847214 15278 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.835908 15279 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:08.848347 15278 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.830116 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:08.831862 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.845860 15202 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:08.813879 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:08.838490 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:08.838933 15599 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.856312 15083 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:08.877683 15690 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.878507 15690 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.882393 15278 ts_tablet_manager.cc:1434] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.065s	user 0.010s	sys 0.000s
I20260709 13:02:08.888697 15278 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:08.861662 15630 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:08.887164 15613 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.896310 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 97566.4us
I20260709 13:02:08.895025 15613 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.870154 15087 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:08.903995 15613 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:08.910341 15630 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:08.991312 15592 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:08.993786 15234 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:08.984428 15630 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.102502 15460 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.120159 15278 tablet_bootstrap.cc:654] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.125082 15624 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.137056 15460 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.172991 15688 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.173538 15688 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.175954 15688 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.176602 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:09.145668 15624 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.197974 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.198892 15161 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:09.211846 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.217733 15690 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:09.229507 15203 leader_election.cc:343] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:09.230141 15203 leader_election.cc:343] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:09.270661 15203 leader_election.cc:343] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.224865 15701 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.296613 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 130542us
I20260709 13:02:09.253408 15592 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.304806 15636 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:09.306875 15701 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.347450 15155 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.347841 15701 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.353397 15617 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.356253 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:09.348433 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:09.357301 15460 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.360311 15279 tablet_bootstrap.cc:654] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.371424 15203 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:02:09.373072 15203 leader_election.cc:343] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.368165 15229 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:09.382592 15630 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.384246 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:09.384543 15630 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.385004 15630 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.385538 15599 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.385993 15599 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.386590 15599 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.384066 15694 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.392966 15630 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.399252 15278 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:09.399793 15278 ts_tablet_manager.cc:1403] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.511s	user 0.018s	sys 0.038s
I20260709 13:02:09.400102 15233 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.452395 15279 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:09.445459 15611 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.453123 15279 ts_tablet_manager.cc:1403] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.617s	user 0.027s	sys 0.060s
I20260709 13:02:09.453109 15611 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.469266 15278 raft_consensus.cc:359] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.470674 15278 raft_consensus.cc:385] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.470942 15278 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.471065 15235 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:09.473935 15126 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:09.471652 15278 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.477689 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:09.478273 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.481387 15161 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.490767 15278 ts_tablet_manager.cc:1434] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.091s	user 0.001s	sys 0.007s
I20260709 13:02:09.491768 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.492328 15086 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
W20260709 13:02:09.493788 15203 leader_election.cc:343] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.493642 15697 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 13:02:09.506542 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 202885us
I20260709 13:02:09.506946 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.507668 15156 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:09.511108 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.511761 15088 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:09.515229 15624 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.516222 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.517076 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:09.517730 15082 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.481868 15700 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:02:09.521847 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:09.522823 15700 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.537154 15700 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.546670 15279 raft_consensus.cc:359] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.547446 15279 raft_consensus.cc:385] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.547725 15279 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.548516 15279 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.550168 15279 ts_tablet_manager.cc:1434] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.001s
I20260709 13:02:09.552098 15630 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.551115 15279 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:09.554944 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:09.561559 15085 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:09.561800 15082 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:09.563045 15611 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.567266 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.567920 15086 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:09.571400 15554 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.580084 15554 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.583576 15554 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.586968 15278 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:09.553313 15599 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.594125 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:09.595398 15611 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.595816 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.595904 15611 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.596591 15611 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.597069 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:09.597747 15086 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.599097 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:09.611910 15128 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:09.615847 15086 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:09.621169 15599 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.623571 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.623857 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:09.624344 15162 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:09.628471 15128 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:09.630055 15706 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.630251 15613 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.630661 15706 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:09.630857 15613 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.631564 15706 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.634317 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:09.635146 15082 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.608116 15694 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.578984 15636 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:02:09.641983 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:09.649931 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:09.658407 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.642241 15694 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.665638 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:09.666996 15278 tablet_bootstrap.cc:654] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.666669 15229 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 0.
I20260709 13:02:09.670243 15086 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:09.653479 15672 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.654003 15696 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.674552 15202 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:09.645471 15599 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:02:09.681805 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:02:09.684077 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55681.2us
I20260709 13:02:09.670588 15694 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.693364 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:09.637053 15710 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:02:09.697139 15460 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:09.699466 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.675558 15696 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.685110 15614 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.686127 15672 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:02:09.713860 15054 leader_election.cc:343] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.708266 15696 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.709199 15614 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.707217 15613 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.697499 15636 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:02:09.728790 15614 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.743489 15279 tablet_bootstrap.cc:654] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.762319 15082 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:09.774544 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:09.775673 15691 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.776237 15691 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:09.784212 15694 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.785682 15237 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:09.744815 15672 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.777223 15691 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.772338 15700 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.789770 15163 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:09.791962 15700 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.794804 15696 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.805421 15697 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:09.795621 15722 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:09.850961 15694 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:09.862757 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:09.863405 15083 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.866358 15278 tablet_bootstrap.cc:492] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:09.867074 15278 ts_tablet_manager.cc:1403] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.280s	user 0.013s	sys 0.008s
I20260709 13:02:09.871477 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:09.872365 15085 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:09.882433 15202 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:09.832202 15611 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:09.887905 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:09.906255 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:09.907436 15161 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:02:09.974428 15080 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:02:09.983024 15203 consensus_peers.cc:597] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:09.982755 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 264971us
I20260709 13:02:09.884224 15630 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:09.987013 15203 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:09.901337 15696 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:09.994355 15706 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:02:10.019286 15203 leader_election.cc:343] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:10.004475 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:02:10.037181 15724 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:10.037366 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:10.037667 15083 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:10.023514 15706 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.024453 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:10.048156 15278 raft_consensus.cc:359] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.056286 15278 raft_consensus.cc:385] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.056641 15278 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.061148 15278 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.046748 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:10.047326 15706 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.063452 15087 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:10.074545 15279 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:10.075116 15279 ts_tablet_manager.cc:1403] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.524s	user 0.046s	sys 0.039s
W20260709 13:02:10.075335 15203 leader_election.cc:343] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:10.083751 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:10.089100 15278 ts_tablet_manager.cc:1434] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.222s	user 0.017s	sys 0.009s
I20260709 13:02:10.091034 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:10.100462 15702 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:10.107323 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:10.107983 15160 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.114403 15126 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:10.096396 15672 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:10.106144 15279 raft_consensus.cc:359] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.121196 15279 raft_consensus.cc:385] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.121740 15279 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.099395 15611 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:10.090024 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:10.125404 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:10.122478 15279 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.127333 15278 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
W20260709 13:02:10.133404 15730 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:02:10.124603 15697 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.124455 15636 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:10.133478 15672 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:10.170933 15672 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.145344 15081 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:10.098536 15614 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.174474 15279 ts_tablet_manager.cc:1434] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.002s
I20260709 13:02:10.179443 15279 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:10.202916 15234 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:10.155815 15697 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.205861 15691 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.214140 15691 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.218261 15691 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.226614 15697 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:10.226971 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:10.227753 15088 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:10.238698 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:10.240192 15614 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.253495 15081 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:02:10.230746 15160 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:10.261096 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:10.290364 15734 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.237287 15739 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.064s
W20260709 13:02:10.307206 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44051us
I20260709 13:02:10.285768 15706 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.309257 15734 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:10.309762 15236 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 2: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:10.310370 15128 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:10.327518 15706 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.325686 15697 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:10.329149 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:10.340232 14985 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:02:10.335119 15734 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.342640 15697 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:10.347892 15697 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.349092 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:10.357968 15711 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:10.361536 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45339.3us
I20260709 13:02:10.369889 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:10.370221 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:10.370765 15084 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 1: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:02:10.394771 15203 leader_election.cc:343] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:10.375701 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:10.401628 15278 tablet_bootstrap.cc:654] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:10.411708 15203 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:10.405450 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:10.431872 15694 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:10.441743 15160 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:02:10.420737 15079 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:10.423956 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:10.459550 15279 tablet_bootstrap.cc:654] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.500116 15708 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:02:10.468654 15203 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:10.502194 15202 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:10.506451 15599 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.507869 15739 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:10.537185 15736 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.533536 15739 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.548111 15739 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:10.565860 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:10.574824 15738 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:02:10.639772 15750 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:02:10.666109 15034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:10.551352 15736 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:10.760023 15752 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:02:10.760183 15711 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:10.765192 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:10.766245 15736 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:10.770304 15739 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.794577 15739 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.794970 15279 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:10.797252 15279 ts_tablet_manager.cc:1403] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.618s	user 0.014s	sys 0.018s
I20260709 13:02:10.800163 15279 raft_consensus.cc:359] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.800926 15279 raft_consensus.cc:385] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.801208 15279 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.801867 15279 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.802440 15278 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.802944 15278 ts_tablet_manager.cc:1403] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.684s	user 0.026s	sys 0.010s
I20260709 13:02:10.803186 15279 ts_tablet_manager.cc:1434] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:02:10.805986 15278 raft_consensus.cc:359] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.806696 15278 raft_consensus.cc:385] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.806970 15278 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.807644 15278 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.810082 15278 ts_tablet_manager.cc:1434] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:10.812074 15278 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:10.811981 15740 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.812685 15740 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.816455 15740 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.819298 15278 tablet_bootstrap.cc:654] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.825263 15278 tablet_bootstrap.cc:492] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.825757 15278 ts_tablet_manager.cc:1403] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:02:10.828430 15278 raft_consensus.cc:359] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.829138 15278 raft_consensus.cc:385] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.829393 15278 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.829972 15278 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.831480 15278 ts_tablet_manager.cc:1434] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:02:10.840402 15278 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:10.847318 15278 tablet_bootstrap.cc:654] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.853430 15278 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.853940 15278 ts_tablet_manager.cc:1403] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:02:10.856938 15278 raft_consensus.cc:359] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.857797 15278 raft_consensus.cc:385] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.858108 15278 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.858776 15278 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.861160 15278 ts_tablet_manager.cc:1434] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:10.862111 15278 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:10.865283 15705 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:10.865980 15705 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.868610 15278 tablet_bootstrap.cc:654] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.868287 15705 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:10.874501 15278 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.874987 15278 ts_tablet_manager.cc:1403] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:02:10.877646 15278 raft_consensus.cc:359] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.878369 15278 raft_consensus.cc:385] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.878643 15278 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.879235 15278 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.881285 15278 ts_tablet_manager.cc:1434] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:02:10.882212 15278 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:10.886572 15278 tablet_bootstrap.cc:654] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.902729 15278 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.903227 15278 ts_tablet_manager.cc:1403] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:02:10.906141 15278 raft_consensus.cc:359] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.906896 15278 raft_consensus.cc:385] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.907197 15278 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.907802 15278 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.910076 15278 ts_tablet_manager.cc:1434] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:10.911038 15278 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:10.917155 15278 tablet_bootstrap.cc:654] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:10.928768 15278 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:10.929342 15278 ts_tablet_manager.cc:1403] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:02:10.957299 15278 raft_consensus.cc:359] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.958058 15278 raft_consensus.cc:385] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:10.958338 15278 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:10.958945 15278 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:10.965583 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:10.966310 15086 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:10.973349 15086 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:11.016700 15157 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.025599 15155 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.049925 14984 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.055056 14984 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.057331 14984 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.059342 14984 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.061458 15202 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:11.065192 15705 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:11.067998 15705 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:11.068521 15705 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:11.069397 15705 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.123917 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:11.138720 15054 leader_election.cc:343] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:11.144604 15694 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:11.145257 15694 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.147786 15694 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:11.166325 15278 ts_tablet_manager.cc:1434] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.237s	user 0.010s	sys 0.000s
I20260709 13:02:11.167783 15278 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Bootstrap starting.
I20260709 13:02:11.187587 15688 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:11.188180 15688 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.194064 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.205997 15688 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:11.211426 15278 tablet_bootstrap.cc:654] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:11.212316 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 72636us
I20260709 13:02:11.249864 15278 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: No bootstrap required, opened a new log
I20260709 13:02:11.251159 15279 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Bootstrap starting.
I20260709 13:02:11.252174 15278 ts_tablet_manager.cc:1403] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.000s
I20260709 13:02:11.255203 15278 raft_consensus.cc:359] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.255898 15278 raft_consensus.cc:385] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:11.256170 15278 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Initialized, Role: FOLLOWER
I20260709 13:02:11.256987 15278 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.259406 15278 ts_tablet_manager.cc:1434] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:02:11.262692 15279 tablet_bootstrap.cc:654] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:11.269093 15279 tablet_bootstrap.cc:492] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: No bootstrap required, opened a new log
I20260709 13:02:11.269596 15279 ts_tablet_manager.cc:1403] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 13:02:11.272495 15279 raft_consensus.cc:359] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.273607 15279 raft_consensus.cc:385] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:11.273898 15279 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Initialized, Role: FOLLOWER
I20260709 13:02:11.274610 15279 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.276665 15279 ts_tablet_manager.cc:1434] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:02:11.284204 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.284816 15088 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:11.285303 15636 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.285794 15636 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.285986 15202 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:11.287106 15739 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:11.287501 15739 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.287846 15739 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.287972 15636 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:11.288527 15636 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.290032 15636 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.294749 15636 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:11.295773 15739 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.298285 15739 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:11.299018 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:11.299527 15088 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.305711 15088 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:11.307003 15202 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:11.314551 15672 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:11.315100 15672 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:11.315977 15672 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.323652 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.324625 15087 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:11.325917 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.329600 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.326807 15087 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 1: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:11.332499 15636 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.333031 15636 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.335182 15636 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:11.337854 15690 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:11.341462 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:11.363413 14986 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 2, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.366757 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:11.367429 15157 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.367901 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:11.368487 15163 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 0.
I20260709 13:02:11.369521 15160 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.366243 14985 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.369400 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:11.372628 15159 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.378371 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:11.381758 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:11.382337 15161 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.391451 15739 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:11.394227 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:11.394896 15237 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 0.
I20260709 13:02:11.404157 15157 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:11.419071 15160 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:11.427153 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:11.423974 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:11.419659 15237 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:11.428197 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:11.411051 15161 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:11.430187 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:11.428689 15734 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:02:11.451783 15672 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:11.471388 15203 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:11.472126 15699 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.477219 15203 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:11.478197 15699 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.479832 15699 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.494371 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:11.495487 15737 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:11.495862 15737 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.496198 15737 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.504369 15737 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.507313 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:11.507908 15161 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.513046 15737 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:11.514163 15161 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:11.514190 15690 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.515282 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:11.516009 15734 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:11.516564 15734 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:11.517433 15734 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:11.518720 15126 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:11.519901 15688 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:11.521157 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:11.522403 15690 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.525154 15126 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:11.527807 15690 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:11.532380 15692 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.544626 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:11.546945 15688 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:11.551489 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:11.552292 15230 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:11.560528 15230 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:11.589051 15740 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.604218 14985 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.605898 14985 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.625110 14986 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.632406 15740 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.720871 15699 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.742049 15740 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:11.980719 15712 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:11.998862 15184 debug-util.cc:398] Leaking SignalData structure 0x7b080017bba0 after lost signal to thread 13970
W20260709 13:02:11.999969 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800096380 after lost signal to thread 15020
W20260709 13:02:12.001165 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08002682c0 after lost signal to thread 15112
W20260709 13:02:12.002228 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08002a76a0 after lost signal to thread 15187
W20260709 13:02:12.003294 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9a80 after lost signal to thread 15261
I20260709 13:02:12.010051 15736 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:12.722865 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:12.788214 15794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:12.816601 13967 server_base.cc:1061] running on GCE node
W20260709 13:02:12.817548 15795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:12.819057 15798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:12.821066 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:12.821478 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:12.821786 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602132821765 us; error 0 us; skew 500 ppm
I20260709 13:02:12.822870 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:12.965447 13967 webserver.cc:533] Webserver started at http://127.13.163.196:39599/ using document root <none> and password file <none>
I20260709 13:02:12.967665 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:12.968127 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:12.968803 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:12.972523 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-3-root/instance:
uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
format_stamp: "Formatted at 2026-07-09 13:02:12 on dist-test-slave-w3b1"
I20260709 13:02:12.989543 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:02:13.073241 15808 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:13.081080 13967 fs_manager.cc:730] Time spent opening block manager: real 0.087s	user 0.001s	sys 0.009s
I20260709 13:02:13.081597 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
format_stamp: "Formatted at 2026-07-09 13:02:12 on dist-test-slave-w3b1"
I20260709 13:02:13.082075 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:02:13.244055 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54557.6us
I20260709 13:02:13.330960 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:13.333209 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:13.339710 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:13.345160 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:13.345631 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:02:13.346416 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:13.346737 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:13.639834 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.196:34641
I20260709 13:02:13.640054 15872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.196:34641 every 8 connection(s)
I20260709 13:02:13.720008 13967 auto_leader_rebalancer-test.cc:384] add a tserver: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:13.732364 15873 heartbeater.cc:344] Connected to a master server at 127.13.163.254:45133
I20260709 13:02:13.733035 15873 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:13.734217 15873 heartbeater.cc:507] Master 127.13.163.254:45133 requested a full tablet report, sending...
I20260709 13:02:13.737311 14985 ts_manager.cc:194] Registered new tserver with Master: 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:13.739404 14985 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50368
I20260709 13:02:14.022555 15231 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:14.069051 15160 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.071009 15617 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.069092 15081 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.087251 15617 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.103987 15617 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:14.151804 15201 consensus_peers.cc:597] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 42a84a565612402dbb559ec973692018. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:14.154140 15082 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.273167 14985 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.355167 15161 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.441844 15082 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:14.453547 15231 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.456207 15809 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.468587 15161 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.472471 15809 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:14.505450 15052 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b358d067cea0428190a484601dea7c69. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:14.523416 15803 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.527953 15157 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.596747 15899 ts_tablet_manager.cc:933] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:14.606703 15082 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:14.609978 14984 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.625128 15231 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.669968 15899 tablet_copy_client.cc:323] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
W20260709 13:02:14.689628 15052 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14bd17b479354abca0014ae059714e8f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:14.713366 15157 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.719306 15232 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.722748 15886 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.731135 15886 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.743414 15873 heartbeater.cc:499] Master 127.13.163.254:45133 was elected leader, sending a full tablet report...
I20260709 13:02:14.749996 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 42a84a565612402dbb559ec973692018 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:14.750674 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 42a84a565612402dbb559ec973692018 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-42a84a565612402dbb559ec973692018
I20260709 13:02:14.764361 15231 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.763172 15712 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.779700 15157 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.792349 15082 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:14.811976 14984 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:14.817344 15247 tablet_copy_source_session.cc:215] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:02:14.821794 15052 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b6a82b88a6f4f6093256a0d85708d37. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:14.825841 15231 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.827562 15159 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:14.830138 15803 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:14.831710 15809 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:14.841507 15899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a84a565612402dbb559ec973692018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:14.868572 15886 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:14.886307 15231 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:15.026569 15157 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:15.027999 14986 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:15.076308 15899 tablet_copy_client.cc:806] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:15.082022 15899 tablet_copy_client.cc:670] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:15.087898 15899 tablet_copy_client.cc:538] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:15.129956 15913 ts_tablet_manager.cc:933] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:15.139560 15899 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:15.146706 15913 tablet_copy_client.cc:323] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:15.190412 15098 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet b358d067cea0428190a484601dea7c69 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:15.191049 15098 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet b358d067cea0428190a484601dea7c69 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-b358d067cea0428190a484601dea7c69
I20260709 13:02:15.198830 15899 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.200125 15098 tablet_copy_source_session.cc:215] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:15.204622 15899 tablet_bootstrap.cc:492] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:15.208851 15917 ts_tablet_manager.cc:933] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:15.209085 15913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b358d067cea0428190a484601dea7c69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:15.208556 15899 ts_tablet_manager.cc:1403] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.011s
I20260709 13:02:15.213586 15899 raft_consensus.cc:359] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.214524 15899 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:15.215332 15899 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.222505 15917 tablet_copy_client.cc:323] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:15.224824 15098 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 14bd17b479354abca0014ae059714e8f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:15.225431 15098 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 14bd17b479354abca0014ae059714e8f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-14bd17b479354abca0014ae059714e8f
I20260709 13:02:15.234511 15098 tablet_copy_source_session.cc:215] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:15.238111 15899 ts_tablet_manager.cc:1434] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.009s
I20260709 13:02:15.238306 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bd17b479354abca0014ae059714e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:15.240495 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-42a84a565612402dbb559ec973692018 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:15.241063 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-42a84a565612402dbb559ec973692018 on tablet 42a84a565612402dbb559ec973692018 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:15.265918 15917 tablet_copy_client.cc:806] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:15.266644 15917 tablet_copy_client.cc:670] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:15.271116 15917 tablet_copy_client.cc:538] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:15.279325 15913 tablet_copy_client.cc:806] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:15.281342 15917 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:15.281659 15913 tablet_copy_client.cc:670] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:15.288642 15913 tablet_copy_client.cc:538] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:15.303361 15913 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:15.325230 15899 ts_tablet_manager.cc:933] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:15.329007 15899 tablet_copy_client.cc:323] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:15.331600 15098 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 6b6a82b88a6f4f6093256a0d85708d37 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:15.332235 15098 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 6b6a82b88a6f4f6093256a0d85708d37 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-6b6a82b88a6f4f6093256a0d85708d37
I20260709 13:02:15.342763 15917 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.344179 15917 tablet_bootstrap.cc:492] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:15.348676 15917 ts_tablet_manager.cc:1403] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.068s	user 0.023s	sys 0.009s
I20260709 13:02:15.361011 15917 raft_consensus.cc:359] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.362059 15917 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:15.362834 15917 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.369081 15917 ts_tablet_manager.cc:1434] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 13:02:15.375758 15098 tablet_copy_source_session.cc:215] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:15.379483 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-14bd17b479354abca0014ae059714e8f received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:15.380075 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-14bd17b479354abca0014ae059714e8f on tablet 14bd17b479354abca0014ae059714e8f with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:15.381841 15899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6a82b88a6f4f6093256a0d85708d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:15.382961 15913 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.383983 15913 tablet_bootstrap.cc:492] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:15.385511 15913 ts_tablet_manager.cc:1403] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.082s	user 0.019s	sys 0.007s
I20260709 13:02:15.388616 15913 raft_consensus.cc:359] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.389799 15913 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:15.390506 15913 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.393133 15913 ts_tablet_manager.cc:1434] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:02:15.397125 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-b358d067cea0428190a484601dea7c69 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:15.397578 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-b358d067cea0428190a484601dea7c69 on tablet b358d067cea0428190a484601dea7c69 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:15.403805 15899 tablet_copy_client.cc:806] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:15.404471 15899 tablet_copy_client.cc:670] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:15.408209 15899 tablet_copy_client.cc:538] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:15.418419 15899 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:15.457507 15899 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.458590 15899 tablet_bootstrap.cc:492] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:15.459396 15899 ts_tablet_manager.cc:1403] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.042s	user 0.032s	sys 0.007s
I20260709 13:02:15.462679 15899 raft_consensus.cc:359] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.463677 15899 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:15.464449 15899 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:15.470741 15899 ts_tablet_manager.cc:1434] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:02:15.475069 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-6b6a82b88a6f4f6093256a0d85708d37 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:15.475565 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-6b6a82b88a6f4f6093256a0d85708d37 on tablet 6b6a82b88a6f4f6093256a0d85708d37 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:15.480564 15848 raft_consensus.cc:1217] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:15.623858 15848 raft_consensus.cc:1217] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:02:15.716861 15848 raft_consensus.cc:1217] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:15.797351 15848 raft_consensus.cc:1217] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:16.034145 15910 raft_consensus.cc:1064] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:16.053543 15910 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:16.204623 15848 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.207813 15891 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:16.258702 15161 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.278263 15082 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.393239 15793 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:16.399931 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44376.1us
W20260709 13:02:16.401990 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 46801us
W20260709 13:02:16.402557 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46799.2us
I20260709 13:02:16.425009 15712 raft_consensus.cc:1064] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:16.427511 15712 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:16.428879 15909 raft_consensus.cc:1064] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:16.433176 15909 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:16.456977 15891 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:16.477174 15809 raft_consensus.cc:1064] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:16.481715 15809 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:16.543510 15157 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:16.545089 15712 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:16.555124 15159 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.557485 15909 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:16.596071 15848 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.598276 15848 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:16.607959 15806 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.648593 15231 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.653111 15231 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:16.667052 15159 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.669661 15932 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:16.678977 15082 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.697901 15848 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.713625 15803 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:16.715531 15803 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:16.722532 15847 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.739462 14986 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:16.777139 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46276.1us
W20260709 13:02:16.817232 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52323.7us
I20260709 13:02:16.822911 15712 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:16.828708 15803 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:16.834944 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59619.7us
I20260709 13:02:16.922353 15909 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.923969 15809 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.925712 15928 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.928992 15848 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.948884 15847 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.931203 15846 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:16.978209 15932 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:17.002910 15231 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:17.093861 14984 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:17.097007 15230 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:17.098747 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79775.5us
I20260709 13:02:17.104085 14985 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:17.108207 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93426.4us
I20260709 13:02:17.115731 15712 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:17.152103 15157 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.125034 15229 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:17.174753 15846 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:02:17.181106 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 71038.8us
I20260709 13:02:17.183858 15237 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.186198 15927 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:17.181605 14984 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:17.193289 15157 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:17.243873 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44524.4us
I20260709 13:02:17.247207 15161 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.257925 15082 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:17.258986 15080 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:17.264534 15846 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:17.265808 15237 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:17.266422 15951 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:17.266940 15846 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:17.267343 15909 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.268283 15932 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.276341 15803 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.277508 15163 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.283823 15928 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.287396 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b358d067cea0428190a484601dea7c69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:17.288671 15846 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.296466 15237 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.307708 15158 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:02:17.309918 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78310.5us
I20260709 13:02:17.314227 15847 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.312494 15926 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.320333 15846 raft_consensus.cc:2955] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.320827 14984 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.331106 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 14bd17b479354abca0014ae059714e8f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:17.341063 15235 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.348053 15160 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:17.355502 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet b358d067cea0428190a484601dea7c69 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
W20260709 13:02:17.356637 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.364034 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 42a84a565612402dbb559ec973692018 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:17.386672 15803 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.395797 14985 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.397615 14985 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.411981 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 14bd17b479354abca0014ae059714e8f with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:17.414563 15163 raft_consensus.cc:2955] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.418282 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 42a84a565612402dbb559ec973692018 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:17.439813 15959 tablet_replica.cc:333] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:17.442982 15959 raft_consensus.cc:2243] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:17.447405 15961 tablet_replica.cc:333] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:17.448400 15960 tablet_replica.cc:333] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:17.457895 15960 raft_consensus.cc:2243] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:17.461320 15960 raft_consensus.cc:2272] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:17.465252 15960 ts_tablet_manager.cc:1916] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:17.456640 15959 raft_consensus.cc:2272] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:17.475476 15959 ts_tablet_manager.cc:1916] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:17.501957 15961 raft_consensus.cc:2243] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:17.502785 15961 raft_consensus.cc:2272] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:17.501746 15948 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:17.506762 15961 ts_tablet_manager.cc:1916] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:17.535390 15080 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:17.541684 15961 ts_tablet_manager.cc:1929] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:17.542795 15960 ts_tablet_manager.cc:1929] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:17.543471 15960 log.cc:1199] T 14bd17b479354abca0014ae059714e8f P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/14bd17b479354abca0014ae059714e8f
I20260709 13:02:17.545601 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 14bd17b479354abca0014ae059714e8f (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:17.548067 15846 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:17.546831 15961 log.cc:1199] T 42a84a565612402dbb559ec973692018 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/42a84a565612402dbb559ec973692018
I20260709 13:02:17.551178 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 42a84a565612402dbb559ec973692018 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:17.554060 15928 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.562919 15163 raft_consensus.cc:2955] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.569476 15928 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.573529 15846 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.589305 15237 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:17.596253 15803 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.601266 15959 ts_tablet_manager.cc:1929] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:02:17.601845 15959 log.cc:1199] T b358d067cea0428190a484601dea7c69 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/b358d067cea0428190a484601dea7c69
I20260709 13:02:17.603780 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet b358d067cea0428190a484601dea7c69 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:17.609226 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b6a82b88a6f4f6093256a0d85708d37 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:17.614830 14985 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.636567 15237 raft_consensus.cc:2955] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:17.646174 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6a82b88a6f4f6093256a0d85708d37 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:17.647554 15960 tablet_replica.cc:333] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:17.667781 15960 raft_consensus.cc:2243] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:17.669044 15960 raft_consensus.cc:2272] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:17.679173 15960 ts_tablet_manager.cc:1916] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:17.697207 15960 ts_tablet_manager.cc:1929] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:17.697646 15960 log.cc:1199] T 6b6a82b88a6f4f6093256a0d85708d37 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/6b6a82b88a6f4f6093256a0d85708d37
I20260709 13:02:17.699321 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 6b6a82b88a6f4f6093256a0d85708d37 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:18.925755 15160 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:18.957525 15230 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:18.959051 15954 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:18.971800 15087 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:18.973412 15952 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:02:19.009421 15127 consensus_peers.cc:597] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efef46e0b3c14af2a7179beba5c160b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:19.035437 15952 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.039176 15082 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.051589 15230 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.059772 15082 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.060081 14985 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:19.098448 15052 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09eea6ae09f04b0b9937f09218fcb3b1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:19.107414 15230 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.108814 15160 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.111371 15803 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.113188 15803 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.137856 15951 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.141597 15158 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.146240 15230 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.177119 14986 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.185775 15230 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:19.238159 15201 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a12ada8057b141518113efeb519c6514. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:19.238484 15087 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.240720 15158 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.241312 15926 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.245594 15926 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.270839 15963 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.282977 15160 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.278499 15087 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.298463 15158 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.301844 14986 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:19.373533 15127 consensus_peers.cc:597] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf60e792da464094a6aa2bfbcc5d71f2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:19.379624 15087 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.384681 15954 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:19.394047 15237 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:19.396816 15952 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:02:19.406157 15984 ts_tablet_manager.cc:933] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:19.411469 15087 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.415199 15984 tablet_copy_client.cc:323] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:19.418358 15230 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.409679 15983 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.436303 14986 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:19.531576 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet efef46e0b3c14af2a7179beba5c160b4 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:19.532249 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet efef46e0b3c14af2a7179beba5c160b4 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-efef46e0b3c14af2a7179beba5c160b4
I20260709 13:02:19.541584 15173 tablet_copy_source_session.cc:215] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:19.545303 15984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efef46e0b3c14af2a7179beba5c160b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:19.572890 15984 tablet_copy_client.cc:806] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:19.573647 15984 tablet_copy_client.cc:670] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:19.584008 15984 tablet_copy_client.cc:538] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:19.601462 15984 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:19.631747 15984 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:19.635229 15984 tablet_bootstrap.cc:492] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:19.635885 15984 ts_tablet_manager.cc:1403] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.000s
I20260709 13:02:19.639120 15984 raft_consensus.cc:359] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.641316 15984 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:19.643798 15984 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.653368 15984 ts_tablet_manager.cc:1434] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:02:19.655169 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-efef46e0b3c14af2a7179beba5c160b4 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:19.655789 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-efef46e0b3c14af2a7179beba5c160b4 on tablet efef46e0b3c14af2a7179beba5c160b4 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:19.668818 15984 ts_tablet_manager.cc:933] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:19.674089 15984 tablet_copy_client.cc:323] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:19.675904 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 09eea6ae09f04b0b9937f09218fcb3b1 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:19.676508 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 09eea6ae09f04b0b9937f09218fcb3b1 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-09eea6ae09f04b0b9937f09218fcb3b1
I20260709 13:02:19.684176 15097 tablet_copy_source_session.cc:215] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:19.689476 15984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09eea6ae09f04b0b9937f09218fcb3b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:19.689813 15992 ts_tablet_manager.cc:933] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:19.693328 15992 tablet_copy_client.cc:323] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:19.695230 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet a12ada8057b141518113efeb519c6514 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:19.695840 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet a12ada8057b141518113efeb519c6514 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-a12ada8057b141518113efeb519c6514
I20260709 13:02:19.706094 15984 tablet_copy_client.cc:806] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:19.706809 15984 tablet_copy_client.cc:670] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:19.719669 15247 tablet_copy_source_session.cc:215] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:19.720095 15984 tablet_copy_client.cc:538] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:19.730902 15984 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:19.795437 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12ada8057b141518113efeb519c6514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:19.888152 15984 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:19.889451 15984 tablet_bootstrap.cc:492] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:19.890272 15984 ts_tablet_manager.cc:1403] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.160s	user 0.067s	sys 0.078s
I20260709 13:02:19.893899 15984 raft_consensus.cc:359] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.894809 15984 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:19.895485 15984 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:19.897864 15984 ts_tablet_manager.cc:1434] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:02:19.899351 15847 raft_consensus.cc:1217] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:19.902251 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-09eea6ae09f04b0b9937f09218fcb3b1 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:19.902740 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-09eea6ae09f04b0b9937f09218fcb3b1 on tablet 09eea6ae09f04b0b9937f09218fcb3b1 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:19.907655 15992 tablet_copy_client.cc:806] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:19.908396 15992 tablet_copy_client.cc:670] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:19.924075 15984 ts_tablet_manager.cc:933] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:19.953953 15984 tablet_copy_client.cc:323] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:19.954878 15992 tablet_copy_client.cc:538] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:19.955858 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet bf60e792da464094a6aa2bfbcc5d71f2 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:19.956468 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet bf60e792da464094a6aa2bfbcc5d71f2 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-bf60e792da464094a6aa2bfbcc5d71f2
I20260709 13:02:19.967756 15992 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:19.991317 15173 tablet_copy_source_session.cc:215] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:20.000726 15984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf60e792da464094a6aa2bfbcc5d71f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:20.021056 15984 tablet_copy_client.cc:806] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:20.025084 15984 tablet_copy_client.cc:670] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:20.028009 15992 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:20.029130 15992 tablet_bootstrap.cc:492] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:20.029887 15992 ts_tablet_manager.cc:1403] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.062s	user 0.021s	sys 0.015s
I20260709 13:02:20.033562 15992 raft_consensus.cc:359] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:20.034559 15992 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:20.036473 15992 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:20.040606 15992 ts_tablet_manager.cc:1434] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:02:20.042800 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-a12ada8057b141518113efeb519c6514 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:20.043293 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-a12ada8057b141518113efeb519c6514 on tablet a12ada8057b141518113efeb519c6514 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:20.051920 15984 tablet_copy_client.cc:538] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:20.062268 15984 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:20.090379 15984 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:20.091418 15984 tablet_bootstrap.cc:492] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:20.092134 15984 ts_tablet_manager.cc:1403] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.008s
I20260709 13:02:20.094811 15984 raft_consensus.cc:359] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:20.095662 15984 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:20.096998 15984 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:20.101395 15984 ts_tablet_manager.cc:1434] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:02:20.107579 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-bf60e792da464094a6aa2bfbcc5d71f2 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:20.108073 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-bf60e792da464094a6aa2bfbcc5d71f2 on tablet bf60e792da464094a6aa2bfbcc5d71f2 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:20.132676 15847 raft_consensus.cc:1217] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:20.291671 15847 raft_consensus.cc:1217] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:20.311056 15983 raft_consensus.cc:1064] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:20.313679 15983 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:20.322305 15237 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.323601 15983 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:20.325277 15087 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.331576 15847 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.334489 15989 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.336454 15989 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.373078 15954 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.380282 15082 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.400449 14984 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.402793 15846 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.406071 15237 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:20.545315 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48033.8us
I20260709 13:02:20.566591 15847 raft_consensus.cc:1217] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:20.581609 15996 raft_consensus.cc:1064] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:20.584254 15996 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:20.605340 15847 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.610780 15160 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.613473 15803 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.615301 15951 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.622017 15230 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.625222 16015 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.642272 15996 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.645578 15230 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.654493 15847 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.664534 15158 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:20.671027 15846 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:20.672874 15954 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:20.676939 15230 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:20.676687 14986 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.681708 16005 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:20.690348 15954 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.692133 15846 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.700374 15158 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.702764 15230 raft_consensus.cc:2955] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.707155 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet efef46e0b3c14af2a7179beba5c160b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:20.725013 14986 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:20.740825 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet efef46e0b3c14af2a7179beba5c160b4 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:20.745074 16008 raft_consensus.cc:1064] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:20.747617 16008 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:20.750301 16021 tablet_replica.cc:333] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:20.751511 16021 raft_consensus.cc:2243] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.752410 16021 raft_consensus.cc:2272] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.759688 16021 ts_tablet_manager.cc:1916] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:20.785682 16021 ts_tablet_manager.cc:1929] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:20.786176 16021 log.cc:1199] T efef46e0b3c14af2a7179beba5c160b4 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/efef46e0b3c14af2a7179beba5c160b4
I20260709 13:02:20.788074 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet efef46e0b3c14af2a7179beba5c160b4 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:20.788450 15082 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:20.794593 15847 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:20.796978 15951 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:20.812386 15848 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.813407 15087 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.815795 15160 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:20.816107 15963 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.817160 16013 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.822664 16013 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:20.837234 15158 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:20.839722 16015 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:20.841665 16015 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.848558 15846 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.849414 15163 raft_consensus.cc:2955] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.891335 14983 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.894553 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 09eea6ae09f04b0b9937f09218fcb3b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:20.928776 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 09eea6ae09f04b0b9937f09218fcb3b1 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:20.916307 15963 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.939230 15087 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.949023 15847 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.967217 16026 tablet_replica.cc:333] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:20.971971 16026 raft_consensus.cc:2243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.977352 16026 raft_consensus.cc:2272] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.978416 15163 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.976585 14983 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:20.997793 16010 raft_consensus.cc:1064] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:21.000325 16010 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:21.019261 16026 ts_tablet_manager.cc:1916] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:21.020161 15229 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:21.030634 15087 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:21.032199 15847 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:21.033838 15962 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:21.035768 15847 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:21.039728 15981 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:21.041885 15082 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:21.044768 15989 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:21.046763 15989 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:21.052536 15237 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:21.062780 15989 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:21.064002 16008 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.089804 16026 ts_tablet_manager.cc:1929] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:21.090399 16026 log.cc:1199] T 09eea6ae09f04b0b9937f09218fcb3b1 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/09eea6ae09f04b0b9937f09218fcb3b1
I20260709 13:02:21.095191 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 09eea6ae09f04b0b9937f09218fcb3b1 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:21.098963 15080 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.104132 15847 raft_consensus.cc:2955] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.111536 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a12ada8057b141518113efeb519c6514 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:21.119488 15846 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.130313 16005 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.140328 14985 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:21.172333 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet a12ada8057b141518113efeb519c6514 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:21.186795 15237 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.193471 14986 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.202243 16032 tablet_replica.cc:333] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:21.214558 15080 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.219220 16032 raft_consensus.cc:2243] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.220077 16032 raft_consensus.cc:2272] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.223268 16032 ts_tablet_manager.cc:1916] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:21.224082 15160 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:21.238514 15237 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:21.239948 15846 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:21.241859 15989 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:21.248384 16005 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:21.255061 16032 ts_tablet_manager.cc:1929] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:21.255620 16032 log.cc:1199] T a12ada8057b141518113efeb519c6514 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/a12ada8057b141518113efeb519c6514
I20260709 13:02:21.257627 16031 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.259119 15847 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.260699 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet a12ada8057b141518113efeb519c6514 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:21.271929 15237 raft_consensus.cc:2955] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.277122 14984 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:21.285408 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf60e792da464094a6aa2bfbcc5d71f2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:21.293929 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet bf60e792da464094a6aa2bfbcc5d71f2 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:21.295003 16021 tablet_replica.cc:333] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:21.296120 16021 raft_consensus.cc:2243] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.296772 16021 raft_consensus.cc:2272] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.302587 16021 ts_tablet_manager.cc:1916] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:21.318589 16021 ts_tablet_manager.cc:1929] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:21.319077 16021 log.cc:1199] T bf60e792da464094a6aa2bfbcc5d71f2 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/bf60e792da464094a6aa2bfbcc5d71f2
I20260709 13:02:21.320657 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet bf60e792da464094a6aa2bfbcc5d71f2 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:22.403928 15082 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:22.411355 15052 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60df009dd21a4cb3867d92cb3a69c1e4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:22.411646 15160 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:22.412983 15237 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:22.412902 15996 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:22.414392 16034 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.422402 15996 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.424203 15237 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.424203 15160 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.443322 14984 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.447141 15160 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:22.460204 15082 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:22.461258 15237 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:22.461671 15127 consensus_peers.cc:597] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13e4a6d3ac0a4d6baf95fb7d5c21b5f7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:22.462579 16011 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:22.463999 16031 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.480346 16031 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.481498 15080 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.497246 15229 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.499185 15163 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:22.514374 15235 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:22.514954 15127 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e777c555e1b4153913b5e6459164454. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:22.516054 16031 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.525987 15080 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:22.527638 16043 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.532418 14986 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.538756 16031 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.543094 15081 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.568928 15230 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:22.572683 14984 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.578979 15229 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.591118 15081 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:22.591615 15201 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08bf5d5f44ec498499e1cb1be55c2283. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:22.592895 15962 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.594290 15163 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:22.595578 15962 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:22.618289 15962 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.620450 15081 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:22.862469 14986 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.937825 15163 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:23.006583 16063 ts_tablet_manager.cc:933] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:23.010195 16064 ts_tablet_manager.cc:933] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:23.013271 16064 tablet_copy_client.cc:323] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:23.015012 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:23.015524 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-13e4a6d3ac0a4d6baf95fb7d5c21b5f7
I20260709 13:02:23.020730 16063 tablet_copy_client.cc:323] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:23.025471 15173 tablet_copy_source_session.cc:215] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:23.028940 16064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:23.033463 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 5e777c555e1b4153913b5e6459164454 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:23.034399 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 5e777c555e1b4153913b5e6459164454 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-5e777c555e1b4153913b5e6459164454
I20260709 13:02:23.036770 16068 ts_tablet_manager.cc:933] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:23.040357 16068 tablet_copy_client.cc:323] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:23.041998 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 60df009dd21a4cb3867d92cb3a69c1e4 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:23.042531 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 60df009dd21a4cb3867d92cb3a69c1e4 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-60df009dd21a4cb3867d92cb3a69c1e4
I20260709 13:02:23.045852 15173 tablet_copy_source_session.cc:215] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:23.049249 16063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e777c555e1b4153913b5e6459164454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:23.058910 15097 tablet_copy_source_session.cc:215] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:23.069290 16063 tablet_copy_client.cc:806] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:23.070210 16063 tablet_copy_client.cc:670] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:23.071519 16068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df009dd21a4cb3867d92cb3a69c1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:23.073642 16064 tablet_copy_client.cc:806] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:23.074512 16064 tablet_copy_client.cc:670] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:23.083258 16064 tablet_copy_client.cc:538] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:23.089931 16063 tablet_copy_client.cc:538] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:23.097283 16064 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:23.108886 16063 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:23.110146 16068 tablet_copy_client.cc:806] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:23.110910 16068 tablet_copy_client.cc:670] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:23.184981 16068 tablet_copy_client.cc:538] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:23.217288 16064 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:23.239744 16072 ts_tablet_manager.cc:933] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:23.254518 16064 tablet_bootstrap.cc:492] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:23.260915 16063 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:23.261618 16064 ts_tablet_manager.cc:1403] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.164s	user 0.024s	sys 0.007s
I20260709 13:02:23.261901 16068 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:23.286751 16063 tablet_bootstrap.cc:492] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:23.284570 16064 raft_consensus.cc:359] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.290068 16063 ts_tablet_manager.cc:1403] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.181s	user 0.027s	sys 0.007s
I20260709 13:02:23.290529 16064 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:23.297323 16064 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.304390 16072 tablet_copy_client.cc:323] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:23.296414 16063 raft_consensus.cc:359] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.306926 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 08bf5d5f44ec498499e1cb1be55c2283 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:23.307695 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 08bf5d5f44ec498499e1cb1be55c2283 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-08bf5d5f44ec498499e1cb1be55c2283
I20260709 13:02:23.306113 16063 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:23.313354 16063 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.329507 16064 ts_tablet_manager.cc:1434] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.006s
I20260709 13:02:23.332088 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-13e4a6d3ac0a4d6baf95fb7d5c21b5f7 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:23.332741 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-13e4a6d3ac0a4d6baf95fb7d5c21b5f7 on tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:23.341439 15247 tablet_copy_source_session.cc:215] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:23.346727 16063 ts_tablet_manager.cc:1434] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.055s	user 0.019s	sys 0.006s
I20260709 13:02:23.359211 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-5e777c555e1b4153913b5e6459164454 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:23.359874 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-5e777c555e1b4153913b5e6459164454 on tablet 5e777c555e1b4153913b5e6459164454 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:23.367511 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bf5d5f44ec498499e1cb1be55c2283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:23.380770 15846 raft_consensus.cc:1217] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:23.397439 16072 tablet_copy_client.cc:806] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:23.398437 16072 tablet_copy_client.cc:670] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:23.406154 16072 tablet_copy_client.cc:538] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:23.407980 16068 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:23.408996 16068 tablet_bootstrap.cc:492] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:23.415323 16068 ts_tablet_manager.cc:1403] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.154s	user 0.028s	sys 0.019s
I20260709 13:02:23.424171 16072 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:23.425884 16068 raft_consensus.cc:359] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.426537 16068 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:23.427148 16068 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.431910 16068 ts_tablet_manager.cc:1434] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:02:23.439754 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-60df009dd21a4cb3867d92cb3a69c1e4 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:23.440237 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-60df009dd21a4cb3867d92cb3a69c1e4 on tablet 60df009dd21a4cb3867d92cb3a69c1e4 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:23.461607 16072 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:23.462723 16072 tablet_bootstrap.cc:492] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:23.464748 16072 ts_tablet_manager.cc:1403] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.007s
I20260709 13:02:23.469099 16072 raft_consensus.cc:359] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.470077 16072 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:23.470826 16072 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:23.473407 16072 ts_tablet_manager.cc:1434] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:02:23.478601 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-08bf5d5f44ec498499e1cb1be55c2283 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:23.479108 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-08bf5d5f44ec498499e1cb1be55c2283 on tablet 08bf5d5f44ec498499e1cb1be55c2283 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:23.547065 15847 raft_consensus.cc:1217] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:23.587421 15847 raft_consensus.cc:1217] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:23.652060 15847 raft_consensus.cc:1217] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:02:23.796798 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1aa0 after lost signal to thread 13970
W20260709 13:02:23.798115 15869 debug-util.cc:398] Leaking SignalData structure 0x7b080016cec0 after lost signal to thread 15020
W20260709 13:02:23.799522 15869 debug-util.cc:398] Leaking SignalData structure 0x7b0800297f60 after lost signal to thread 15112
W20260709 13:02:23.800874 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecba0 after lost signal to thread 15187
W20260709 13:02:23.802352 15869 debug-util.cc:398] Leaking SignalData structure 0x7b080022a940 after lost signal to thread 15261
W20260709 13:02:23.803861 15869 debug-util.cc:398] Leaking SignalData structure 0x7b0800319080 after lost signal to thread 15872
I20260709 13:02:23.918673 16065 raft_consensus.cc:1064] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:23.924675 16065 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:23.946921 15847 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:23.948697 16043 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:23.969201 15079 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:23.988911 16065 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.038717 16065 raft_consensus.cc:1064] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:24.045385 15235 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.046012 16065 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.067502 15846 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.070138 15079 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.068990 16065 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:24.072643 16031 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.095777 16043 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.119442 15848 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.137851 15230 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.153848 15079 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.155566 16058 raft_consensus.cc:1064] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:24.158303 16058 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.162783 16055 raft_consensus.cc:1064] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:24.165870 16055 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.198360 15847 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.199872 16052 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.202023 16065 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.211723 14986 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.214555 15084 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.217519 16058 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.223704 15160 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.225421 15996 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.236989 15847 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.241545 16015 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.258042 15163 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.280238 15845 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.285691 16031 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:24.294764 16031 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.296271 15845 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.313822 16065 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.321957 14986 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.332816 15084 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.350572 15846 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.351329 15962 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.354341 15160 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.352133 16015 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.362615 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e777c555e1b4153913b5e6459164454 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:24.392980 16097 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.411975 14986 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.424438 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5e777c555e1b4153913b5e6459164454 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:02:24.429920 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57358.3us
I20260709 13:02:24.432641 15082 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.437919 15087 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.439399 15846 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.504240 14984 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:24.506804 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74331.6us
I20260709 13:02:24.508123 15846 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.477764 16031 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 13:02:24.529461 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68613.5us
I20260709 13:02:24.546144 14984 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.559672 15083 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.572566 15230 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.574285 15845 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.591396 15996 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:02:24.593484 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79811.4us
I20260709 13:02:24.598378 16103 tablet_replica.cc:333] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:24.600919 15160 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.610805 16103 raft_consensus.cc:2243] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.611734 16103 raft_consensus.cc:2272] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.614096 15083 raft_consensus.cc:2955] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.617386 16089 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:24.621189 16103 ts_tablet_manager.cc:1916] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:24.622439 15846 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.619208 16089 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.640668 14984 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.647053 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60df009dd21a4cb3867d92cb3a69c1e4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:24.659276 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 60df009dd21a4cb3867d92cb3a69c1e4 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:24.658924 16103 ts_tablet_manager.cc:1929] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260709 13:02:24.660391 16103 log.cc:1199] T 5e777c555e1b4153913b5e6459164454 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/5e777c555e1b4153913b5e6459164454
I20260709 13:02:24.660943 15163 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:24.662225 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 5e777c555e1b4153913b5e6459164454 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:24.662537 16103 tablet_replica.cc:333] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:24.663487 16103 raft_consensus.cc:2243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.664265 16103 raft_consensus.cc:2272] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.665733 15160 raft_consensus.cc:2955] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.667414 16103 ts_tablet_manager.cc:1916] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:24.684813 16103 ts_tablet_manager.cc:1929] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260709 13:02:24.685454 16103 log.cc:1199] T 60df009dd21a4cb3867d92cb3a69c1e4 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/60df009dd21a4cb3867d92cb3a69c1e4
I20260709 13:02:24.687062 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 60df009dd21a4cb3867d92cb3a69c1e4 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
W20260709 13:02:24.794920 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.821780 16099 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.854732 15229 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.859992 15163 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.866272 15846 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.868695 16104 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:24.868745 15235 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:24.876423 16098 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.880156 15158 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.883791 15846 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.884027 15083 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.889817 15846 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.891683 16107 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:24.898145 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:24.898623 15228 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.904299 16101 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:24.907546 16056 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.908447 16098 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:24.916594 15845 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.914651 14983 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.921708 15229 raft_consensus.cc:2955] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.931041 15083 raft_consensus.cc:2955] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.941538 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:24.961473 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08bf5d5f44ec498499e1cb1be55c2283 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:24.964777 16112 tablet_replica.cc:333] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:24.973713 16112 raft_consensus.cc:2243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.977151 16112 raft_consensus.cc:2272] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.977008 14986 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:24.982582 16112 ts_tablet_manager.cc:1916] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:25.003602 16112 ts_tablet_manager.cc:1929] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:25.004261 16112 log.cc:1199] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/13e4a6d3ac0a4d6baf95fb7d5c21b5f7
I20260709 13:02:25.006102 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:25.012672 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 08bf5d5f44ec498499e1cb1be55c2283 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:25.037014 16113 tablet_replica.cc:333] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:25.047086 16113 raft_consensus.cc:2243] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:25.048108 16113 raft_consensus.cc:2272] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:25.065397 16113 ts_tablet_manager.cc:1916] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:25.084498 16113 ts_tablet_manager.cc:1929] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:25.084971 16113 log.cc:1199] T 08bf5d5f44ec498499e1cb1be55c2283 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/08bf5d5f44ec498499e1cb1be55c2283
I20260709 13:02:25.086866 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 08bf5d5f44ec498499e1cb1be55c2283 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:26.141085 15087 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:26.148092 15052 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d8fe0d92f6f47dfb304a24bf915e2db. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:26.149225 15229 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.149884 15158 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.150547 16089 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.151643 16114 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.171633 15229 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.169538 16114 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.175160 15162 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.190465 15162 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.197238 14984 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:26.216114 15127 consensus_peers.cc:597] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b4be666e4a740bd853d8f2748ad6191. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:26.217317 15087 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.220886 16106 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.222954 15229 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.224427 16106 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.311743 16127 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.329370 15082 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.351770 15083 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.372608 15232 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.383522 14985 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:26.409168 15052 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 808dd81f1b604329848aa94e3ecedc36. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:26.411851 15162 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.413425 16114 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.424000 15232 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.427242 16114 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.442080 16108 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.449975 15232 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.450935 15162 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.463486 15163 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.473690 14984 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.480772 15083 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.483345 16128 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:26.493212 15127 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fd68ef635784c42a7cc0ce9f2fae0f5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:26.507102 15235 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:26.512084 16128 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.518157 16128 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.523653 15082 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.524322 15235 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:26.539067 14984 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.684782 16139 ts_tablet_manager.cc:933] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:26.689224 16140 ts_tablet_manager.cc:933] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:26.693585 16139 tablet_copy_client.cc:323] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:26.702239 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 2b4be666e4a740bd853d8f2748ad6191 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:26.702943 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 2b4be666e4a740bd853d8f2748ad6191 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-2b4be666e4a740bd853d8f2748ad6191
I20260709 13:02:26.737696 15173 tablet_copy_source_session.cc:215] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:26.743232 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4be666e4a740bd853d8f2748ad6191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:26.753691 16140 tablet_copy_client.cc:323] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:26.756783 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 9d8fe0d92f6f47dfb304a24bf915e2db from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:26.757441 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 9d8fe0d92f6f47dfb304a24bf915e2db from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-9d8fe0d92f6f47dfb304a24bf915e2db
I20260709 13:02:26.764099 16139 tablet_copy_client.cc:806] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:26.764909 16139 tablet_copy_client.cc:670] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:26.769506 16139 tablet_copy_client.cc:538] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:26.772796 15097 tablet_copy_source_session.cc:215] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:26.776523 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8fe0d92f6f47dfb304a24bf915e2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:26.780274 16139 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:26.810976 16140 tablet_copy_client.cc:806] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:26.811614 16139 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:26.813067 16139 tablet_bootstrap.cc:492] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:26.813983 16139 ts_tablet_manager.cc:1403] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.013s
I20260709 13:02:26.817693 16139 raft_consensus.cc:359] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.811640 16140 tablet_copy_client.cc:670] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:26.819151 16139 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:26.820024 16139 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.838142 16139 ts_tablet_manager.cc:1434] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.017s
I20260709 13:02:26.840569 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-2b4be666e4a740bd853d8f2748ad6191 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:26.841101 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-2b4be666e4a740bd853d8f2748ad6191 on tablet 2b4be666e4a740bd853d8f2748ad6191 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:26.856159 16140 tablet_copy_client.cc:538] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:26.882787 16140 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:26.904784 16140 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:26.906064 16140 tablet_bootstrap.cc:492] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:26.906857 16140 ts_tablet_manager.cc:1403] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.004s
I20260709 13:02:26.910179 16140 raft_consensus.cc:359] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.911273 16140 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:26.913187 16140 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:26.915980 16140 ts_tablet_manager.cc:1434] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:02:26.923933 16139 ts_tablet_manager.cc:933] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:26.926074 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-9d8fe0d92f6f47dfb304a24bf915e2db received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:26.926548 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-9d8fe0d92f6f47dfb304a24bf915e2db on tablet 9d8fe0d92f6f47dfb304a24bf915e2db with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:26.961277 16139 tablet_copy_client.cc:323] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:26.963124 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:26.963651 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-2fd68ef635784c42a7cc0ce9f2fae0f5
I20260709 13:02:26.971954 15173 tablet_copy_source_session.cc:215] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:26.975329 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:27.001441 16139 tablet_copy_client.cc:806] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:27.002156 16139 tablet_copy_client.cc:670] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:27.006450 16139 tablet_copy_client.cc:538] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:27.015918 16139 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:27.044997 16140 ts_tablet_manager.cc:933] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:27.045630 16139 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:27.047672 16139 tablet_bootstrap.cc:492] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:27.050799 16139 ts_tablet_manager.cc:1403] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.010s
I20260709 13:02:27.055533 16139 raft_consensus.cc:359] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:27.056447 16139 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:27.057158 16139 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:27.061705 16139 ts_tablet_manager.cc:1434] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:02:27.062127 16140 tablet_copy_client.cc:323] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:27.064124 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-2fd68ef635784c42a7cc0ce9f2fae0f5 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:27.064544 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-2fd68ef635784c42a7cc0ce9f2fae0f5 on tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:27.065799 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 808dd81f1b604329848aa94e3ecedc36 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:27.066386 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 808dd81f1b604329848aa94e3ecedc36 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-808dd81f1b604329848aa94e3ecedc36
I20260709 13:02:27.170066 15846 raft_consensus.cc:1217] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:27.214015 15097 tablet_copy_source_session.cc:215] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:27.234339 15846 raft_consensus.cc:1217] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:27.238752 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 808dd81f1b604329848aa94e3ecedc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:27.260272 16140 tablet_copy_client.cc:806] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:27.261003 16140 tablet_copy_client.cc:670] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:27.267293 16140 tablet_copy_client.cc:538] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:27.291029 16140 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:27.322489 16140 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:27.323578 16140 tablet_bootstrap.cc:492] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:27.324405 16140 ts_tablet_manager.cc:1403] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.002s
I20260709 13:02:27.329300 16140 raft_consensus.cc:359] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:27.330341 16140 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:27.331152 16140 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:27.334357 16140 ts_tablet_manager.cc:1434] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:02:27.340564 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-808dd81f1b604329848aa94e3ecedc36 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:27.341027 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-808dd81f1b604329848aa94e3ecedc36 on tablet 808dd81f1b604329848aa94e3ecedc36 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:27.377075 15846 raft_consensus.cc:1217] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:27.607156 16128 raft_consensus.cc:1064] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:27.610440 16128 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:27.625744 15845 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.615756 15846 raft_consensus.cc:1217] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:27.628400 15232 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.630048 15082 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.631587 16104 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:27.632491 16134 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:27.637025 16128 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:27.652859 16134 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.656879 15082 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.678735 15232 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.681890 14984 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.699154 16114 raft_consensus.cc:1064] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:27.701846 16114 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:27.710080 15846 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.731113 15235 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.742837 16089 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:27.780473 15163 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.789738 15846 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:27.794225 16089 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:27.817703 16089 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:27.846582 16108 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.853364 15163 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.886242 15846 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:27.890491 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 56826.2us
I20260709 13:02:27.910719 14986 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.938515 15235 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.936690 15163 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:27.951838 15087 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:27.963155 15158 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:27.968555 16156 raft_consensus.cc:1064] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:27.971170 16156 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:27.972515 15848 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:27.973155 16108 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:27.974260 16134 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:27.984524 15229 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:27.982443 16134 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.988041 15848 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:27.995235 16164 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.000770 16128 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:27.997377 15158 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.010624 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9d8fe0d92f6f47dfb304a24bf915e2db with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:28.008366 15234 raft_consensus.cc:2955] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.016753 14986 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.018816 15845 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.023846 16108 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:28.031651 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2b4be666e4a740bd853d8f2748ad6191 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:28.039747 15847 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.041898 16104 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.064292 15229 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.075286 14982 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.060488 15083 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.059408 15846 raft_consensus.cc:2955] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.089943 16134 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.096043 16156 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.109126 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 2b4be666e4a740bd853d8f2748ad6191 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:28.113314 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d8fe0d92f6f47dfb304a24bf915e2db with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:28.117552 16172 tablet_replica.cc:333] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:28.118970 16172 raft_consensus.cc:2243] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:28.119812 16172 raft_consensus.cc:2272] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:28.123648 16172 ts_tablet_manager.cc:1916] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:28.145658 16134 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.153437 15848 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.173156 16171 tablet_replica.cc:333] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:28.174400 16171 raft_consensus.cc:2243] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:28.175092 16171 raft_consensus.cc:2272] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:28.184199 16171 ts_tablet_manager.cc:1916] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:28.208748 15235 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.210582 16114 raft_consensus.cc:1064] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:28.213240 16114 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:28.218873 14981 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.228926 15083 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.245441 15846 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.248992 15235 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.269253 16164 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.276451 16108 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.287811 15158 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:28.299786 15083 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.301878 16128 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:02:28.300304 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48422.7us
I20260709 13:02:28.305665 15846 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.305689 15158 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:28.308825 16134 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:28.309994 16164 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:28.324494 16172 ts_tablet_manager.cc:1929] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:28.326933 16164 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.328827 16172 log.cc:1199] T 9d8fe0d92f6f47dfb304a24bf915e2db P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/9d8fe0d92f6f47dfb304a24bf915e2db
I20260709 13:02:28.334601 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 9d8fe0d92f6f47dfb304a24bf915e2db (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:28.338338 16171 ts_tablet_manager.cc:1929] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:28.339037 16171 log.cc:1199] T 2b4be666e4a740bd853d8f2748ad6191 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/2b4be666e4a740bd853d8f2748ad6191
I20260709 13:02:28.340814 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 2b4be666e4a740bd853d8f2748ad6191 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:28.350428 16156 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.362578 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:28.365703 15848 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.376981 15235 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.382071 15084 raft_consensus.cc:2955] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.380796 14981 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.422116 15162 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.428395 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:28.429585 16172 tablet_replica.cc:333] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:28.434312 16172 raft_consensus.cc:2243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:28.435321 16172 raft_consensus.cc:2272] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:28.432399 14982 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:28.523953 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 106845us
I20260709 13:02:28.535104 16172 ts_tablet_manager.cc:1916] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:02:28.542322 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100568us
I20260709 13:02:28.559784 16172 ts_tablet_manager.cc:1929] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:28.560355 16172 log.cc:1199] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/2fd68ef635784c42a7cc0ce9f2fae0f5
I20260709 13:02:28.567848 15848 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.562742 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 2fd68ef635784c42a7cc0ce9f2fae0f5 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:28.663066 15083 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:28.669420 15846 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.671149 16174 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:28.679256 15234 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.681337 16179 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:28.693853 16164 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.695326 15846 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.711694 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 808dd81f1b604329848aa94e3ecedc36 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:28.722194 14983 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.729908 15237 raft_consensus.cc:2955] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:28.743106 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 808dd81f1b604329848aa94e3ecedc36 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:28.767748 16181 tablet_replica.cc:333] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:28.768600 16181 raft_consensus.cc:2243] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:28.769212 16181 raft_consensus.cc:2272] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:28.776994 16181 ts_tablet_manager.cc:1916] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:28.792302 16181 ts_tablet_manager.cc:1929] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:28.792691 16181 log.cc:1199] T 808dd81f1b604329848aa94e3ecedc36 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/808dd81f1b604329848aa94e3ecedc36
I20260709 13:02:28.794327 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 808dd81f1b604329848aa94e3ecedc36 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:29.839851 15084 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:29.847774 15237 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:29.848263 15052 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eaa49b1ae77944d7b6866e819d63d466. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:29.850561 15158 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:29.850632 16108 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:29.853601 16164 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.866400 16164 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:29.872629 15158 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:29.883414 15235 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:29.889215 14983 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:29.892423 15237 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:29.929057 15201 consensus_peers.cc:597] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 365e6a51b3034d0483fdd8fc4cd362ea. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:29.933403 15158 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:29.934846 15084 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:29.935027 16052 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.936535 16126 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.955487 16166 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:29.957762 15158 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:29.958984 15084 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:29.979632 15163 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:29.983541 14981 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:29.997134 15127 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf9063404f3d4db9b81ce66060eee5e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:29.998730 15235 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:30.001920 15084 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:30.001518 16134 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:30.005760 16134 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:30.034443 16104 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.035864 15235 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.038990 15084 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.055491 15234 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.065337 15158 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:30.066293 15201 consensus_peers.cc:597] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f14609939d04f1e90837cb17fe19457. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:30.066720 16052 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:30.072008 15087 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:30.071637 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.075078 16052 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:30.115077 16126 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.120736 15163 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.136133 15083 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.158135 14981 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:30.353428 16200 ts_tablet_manager.cc:933] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:30.373020 16200 tablet_copy_client.cc:323] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:30.377921 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 365e6a51b3034d0483fdd8fc4cd362ea from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:30.378499 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 365e6a51b3034d0483fdd8fc4cd362ea from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-365e6a51b3034d0483fdd8fc4cd362ea
I20260709 13:02:30.386060 15247 tablet_copy_source_session.cc:215] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:30.389725 16200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365e6a51b3034d0483fdd8fc4cd362ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:30.405540 16200 tablet_copy_client.cc:806] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:30.406210 16200 tablet_copy_client.cc:670] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:30.411170 16200 tablet_copy_client.cc:538] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:30.417976 16200 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:30.451061 16204 ts_tablet_manager.cc:933] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:30.454501 16204 tablet_copy_client.cc:323] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:30.456470 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet eaa49b1ae77944d7b6866e819d63d466 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:30.457113 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet eaa49b1ae77944d7b6866e819d63d466 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-eaa49b1ae77944d7b6866e819d63d466
I20260709 13:02:30.460512 16200 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:30.461421 16200 tablet_bootstrap.cc:492] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:30.462121 16200 ts_tablet_manager.cc:1403] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.018s
I20260709 13:02:30.464602 15097 tablet_copy_source_session.cc:215] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:30.465507 16200 raft_consensus.cc:359] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.466464 16200 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:30.467200 16200 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.479146 16204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa49b1ae77944d7b6866e819d63d466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:30.480011 16200 ts_tablet_manager.cc:1434] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.006s
I20260709 13:02:30.481827 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-365e6a51b3034d0483fdd8fc4cd362ea received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:30.482285 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-365e6a51b3034d0483fdd8fc4cd362ea on tablet 365e6a51b3034d0483fdd8fc4cd362ea with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:30.497834 16204 tablet_copy_client.cc:806] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:30.498507 16204 tablet_copy_client.cc:670] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:30.502384 16204 tablet_copy_client.cc:538] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:30.512075 16204 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:30.524210 16200 ts_tablet_manager.cc:933] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:30.532186 16200 tablet_copy_client.cc:323] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:30.534199 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 7f14609939d04f1e90837cb17fe19457 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:30.534699 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 7f14609939d04f1e90837cb17fe19457 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-7f14609939d04f1e90837cb17fe19457
I20260709 13:02:30.541365 15247 tablet_copy_source_session.cc:215] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:30.542716 16204 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:30.544732 16204 tablet_bootstrap.cc:492] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:30.545501 16204 ts_tablet_manager.cc:1403] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.034s	user 0.023s	sys 0.000s
I20260709 13:02:30.548035 16200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f14609939d04f1e90837cb17fe19457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:30.549557 16204 raft_consensus.cc:359] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.550534 16204 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:30.551337 16204 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.554209 16204 ts_tablet_manager.cc:1434] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:30.556137 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-eaa49b1ae77944d7b6866e819d63d466 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:30.556604 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-eaa49b1ae77944d7b6866e819d63d466 on tablet eaa49b1ae77944d7b6866e819d63d466 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:30.568396 16200 tablet_copy_client.cc:806] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:30.569037 16200 tablet_copy_client.cc:670] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:30.572829 16200 tablet_copy_client.cc:538] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:30.582502 16200 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:30.610146 16204 ts_tablet_manager.cc:933] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:30.611543 16200 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:30.612573 16200 tablet_bootstrap.cc:492] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:30.613353 16200 ts_tablet_manager.cc:1403] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.031s	user 0.025s	sys 0.001s
I20260709 13:02:30.618589 16200 raft_consensus.cc:359] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.619985 16200 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:30.620945 16200 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.621821 16204 tablet_copy_client.cc:323] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:30.623719 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet cf9063404f3d4db9b81ce66060eee5e7 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:30.624383 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet cf9063404f3d4db9b81ce66060eee5e7 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-cf9063404f3d4db9b81ce66060eee5e7
I20260709 13:02:30.634093 15173 tablet_copy_source_session.cc:215] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:30.623660 16200 ts_tablet_manager.cc:1434] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:02:30.638484 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-7f14609939d04f1e90837cb17fe19457 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:30.638976 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-7f14609939d04f1e90837cb17fe19457 on tablet 7f14609939d04f1e90837cb17fe19457 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:30.643410 16204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9063404f3d4db9b81ce66060eee5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:30.662698 16204 tablet_copy_client.cc:806] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:30.663412 16204 tablet_copy_client.cc:670] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:30.687964 16204 tablet_copy_client.cc:538] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:30.697899 16204 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:30.720507 16204 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:30.721617 16204 tablet_bootstrap.cc:492] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:30.721784 15848 raft_consensus.cc:1217] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:30.722437 16204 ts_tablet_manager.cc:1403] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.005s
I20260709 13:02:30.726300 16204 raft_consensus.cc:359] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.727262 16204 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:30.728420 16204 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:30.731244 16204 ts_tablet_manager.cc:1434] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:30.733119 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-cf9063404f3d4db9b81ce66060eee5e7 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:30.733564 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-cf9063404f3d4db9b81ce66060eee5e7 on tablet cf9063404f3d4db9b81ce66060eee5e7 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:30.991961 15848 raft_consensus.cc:1217] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:31.016913 15848 raft_consensus.cc:1217] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:31.058068 15848 raft_consensus.cc:1217] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:31.270222 16126 raft_consensus.cc:1064] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:31.275187 16126 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.285735 15848 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.286996 15158 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.287348 15083 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.288035 16126 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.289007 16216 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.289924 16166 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.309999 16216 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.313266 15848 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.319706 15158 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.324227 15083 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.330866 14981 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:31.378954 15235 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.387041 16214 raft_consensus.cc:1064] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:31.397922 16214 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.402896 15846 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.404632 16216 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.407169 15846 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.411739 16213 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.412302 16126 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.415297 15158 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.416239 15848 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.417152 16216 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:31.421838 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 365e6a51b3034d0483fdd8fc4cd362ea with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:31.418414 15235 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.425495 16182 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.430616 15083 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.436072 16213 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.476575 14985 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:31.481707 15158 raft_consensus.cc:2955] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.483623 16134 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.485530 15235 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.493577 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 365e6a51b3034d0483fdd8fc4cd362ea with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:31.495282 15846 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.512111 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:31.557058 16227 tablet_replica.cc:333] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:31.558467 16227 raft_consensus.cc:2243] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:31.559285 16227 raft_consensus.cc:2272] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:31.563446 16184 raft_consensus.cc:1064] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:31.568279 15084 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.569083 16184 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.570081 16126 raft_consensus.cc:1064] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:31.572391 16126 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.577150 16227 ts_tablet_manager.cc:1916] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:31.584542 15163 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.590740 15848 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.597117 15087 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.602169 16227 ts_tablet_manager.cc:1929] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:31.602722 16227 log.cc:1199] T 365e6a51b3034d0483fdd8fc4cd362ea P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/365e6a51b3034d0483fdd8fc4cd362ea
I20260709 13:02:31.604398 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 365e6a51b3034d0483fdd8fc4cd362ea (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:31.599869 16214 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:02:31.605985 16134 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:31.611668 15847 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.614674 16164 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.617492 15163 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.626592 16212 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.626906 15235 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.629312 15848 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.630772 16216 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:31.640408 15163 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.632664 16212 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:31.642190 16202 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:31.644913 15087 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:31.654580 16182 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.664412 15083 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.657323 15846 raft_consensus.cc:2955] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.690287 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cf9063404f3d4db9b81ce66060eee5e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:31.703977 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.726253 16184 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.729799 15160 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.739180 15847 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.750834 14983 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.761457 16236 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.763083 15163 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.785353 15235 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.786868 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet cf9063404f3d4db9b81ce66060eee5e7 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:31.788806 16166 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:31.790824 14983 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.791908 15847 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.818822 15234 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.834095 16237 tablet_replica.cc:333] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:31.841172 15846 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.842569 15163 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.843178 16202 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.844015 16216 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:31.869434 15087 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:31.868383 15083 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.876564 16216 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.880797 15235 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.882475 16184 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.890359 15160 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.891230 15847 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:31.891396 16212 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.899336 15847 raft_consensus.cc:2955] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.900686 15235 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.903761 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f14609939d04f1e90837cb17fe19457 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:31.912163 16237 raft_consensus.cc:2243] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:31.911548 16184 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.913357 16237 raft_consensus.cc:2272] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:31.920776 14985 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:31.925143 14970 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 7f14609939d04f1e90837cb17fe19457 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:02:31.928222 16237 ts_tablet_manager.cc:1916] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:31.921753 15846 raft_consensus.cc:2955] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.940346 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f14609939d04f1e90837cb17fe19457 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:31.952977 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eaa49b1ae77944d7b6866e819d63d466 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:31.960918 14985 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:31.965822 16227 tablet_replica.cc:333] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:31.966931 16227 raft_consensus.cc:2243] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:31.967712 16227 raft_consensus.cc:2272] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:31.971239 16227 ts_tablet_manager.cc:1916] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:31.973373 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet eaa49b1ae77944d7b6866e819d63d466 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:31.993355 16237 ts_tablet_manager.cc:1929] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:31.993855 16237 log.cc:1199] T cf9063404f3d4db9b81ce66060eee5e7 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/cf9063404f3d4db9b81ce66060eee5e7
I20260709 13:02:31.995895 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet cf9063404f3d4db9b81ce66060eee5e7 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:31.998242 16242 tablet_replica.cc:333] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:32.003487 16242 raft_consensus.cc:2243] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:32.004146 16242 raft_consensus.cc:2272] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:32.010715 16242 ts_tablet_manager.cc:1916] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:32.013585 16227 ts_tablet_manager.cc:1929] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:32.014014 16227 log.cc:1199] T 7f14609939d04f1e90837cb17fe19457 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/7f14609939d04f1e90837cb17fe19457
I20260709 13:02:32.015770 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 7f14609939d04f1e90837cb17fe19457 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:32.024557 16242 ts_tablet_manager.cc:1929] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:32.025207 16242 log.cc:1199] T eaa49b1ae77944d7b6866e819d63d466 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/eaa49b1ae77944d7b6866e819d63d466
I20260709 13:02:32.026979 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet eaa49b1ae77944d7b6866e819d63d466 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:33.045692 15235 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.051162 15160 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:33.051213 15084 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:33.051888 15201 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06f7ae7c1d614611bb074a65fcc9a959. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:33.052829 16202 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.053737 16246 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.061026 16245 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.062626 15160 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.063102 15084 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.079530 14983 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.086692 15158 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:33.107508 15127 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0afe5ac407a04216b625837a5bca525f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:33.108247 15084 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:33.108229 15235 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:33.109776 16244 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:33.110483 16134 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:33.119133 16134 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.126425 15235 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.134660 15229 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.137058 14985 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:33.143390 15201 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a5f358b91ee427880ebecaca8118e4f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:33.148574 15084 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.151986 15158 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:33.152930 15087 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:33.153616 16202 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.156018 16202 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.183925 16246 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.189971 15160 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.195322 15087 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.355405 15229 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.410352 14982 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:33.476440 15201 consensus_peers.cc:597] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec09a7a4d59c4296aa02dde6bd37f81c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:33.483804 15158 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:33.486620 16272 ts_tablet_manager.cc:933] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:33.488219 16216 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.498582 16272 tablet_copy_client.cc:323] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
W20260709 13:02:33.501827 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 65062.1us
I20260709 13:02:33.502988 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 06f7ae7c1d614611bb074a65fcc9a959 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:33.503731 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 06f7ae7c1d614611bb074a65fcc9a959 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-06f7ae7c1d614611bb074a65fcc9a959
I20260709 13:02:33.505885 16245 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.507869 15158 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.521507 15087 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:33.524765 15247 tablet_copy_source_session.cc:215] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:33.528913 16272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7ae7c1d614611bb074a65fcc9a959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:33.528658 14982 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.531733 16216 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.541706 16275 ts_tablet_manager.cc:933] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:33.556576 16272 tablet_copy_client.cc:806] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:33.558419 16272 tablet_copy_client.cc:670] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:33.558887 16275 tablet_copy_client.cc:323] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:33.566716 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 0a5f358b91ee427880ebecaca8118e4f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:33.567355 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 0a5f358b91ee427880ebecaca8118e4f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-0a5f358b91ee427880ebecaca8118e4f
I20260709 13:02:33.571347 15084 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:33.580359 15247 tablet_copy_source_session.cc:215] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:33.581229 16272 tablet_copy_client.cc:538] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:33.593223 16275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5f358b91ee427880ebecaca8118e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:33.600978 16272 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:33.614044 16278 ts_tablet_manager.cc:933] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:33.617607 16275 tablet_copy_client.cc:806] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:33.618389 16275 tablet_copy_client.cc:670] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:33.623077 16275 tablet_copy_client.cc:538] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:33.628799 16278 tablet_copy_client.cc:323] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:33.632354 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 0afe5ac407a04216b625837a5bca525f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:33.632968 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 0afe5ac407a04216b625837a5bca525f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-0afe5ac407a04216b625837a5bca525f
I20260709 13:02:33.635917 16272 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:33.637238 16272 tablet_bootstrap.cc:492] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:33.638062 16272 ts_tablet_manager.cc:1403] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.007s
I20260709 13:02:33.636047 16275 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:33.643692 15173 tablet_copy_source_session.cc:215] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:33.647396 16278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afe5ac407a04216b625837a5bca525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:33.642450 16272 raft_consensus.cc:359] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.651744 16272 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:33.652590 16272 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:33.824239 16275 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.135s	user 0.017s	sys 0.020s
I20260709 13:02:33.829485 16275 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:33.841385 16275 tablet_bootstrap.cc:492] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:33.847772 16275 ts_tablet_manager.cc:1403] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.212s	user 0.035s	sys 0.040s
I20260709 13:02:33.850427 16275 raft_consensus.cc:359] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.851095 16275 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:33.851755 16275 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:33.867326 16275 ts_tablet_manager.cc:1434] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:02:33.870782 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:33.885154 16272 ts_tablet_manager.cc:1434] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.247s	user 0.019s	sys 0.063s
I20260709 13:02:33.885795 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-0a5f358b91ee427880ebecaca8118e4f received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:33.886312 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-0a5f358b91ee427880ebecaca8118e4f on tablet 0a5f358b91ee427880ebecaca8118e4f with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:33.889843 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-06f7ae7c1d614611bb074a65fcc9a959 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:33.890244 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-06f7ae7c1d614611bb074a65fcc9a959 on tablet 06f7ae7c1d614611bb074a65fcc9a959 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:33.919608 16278 tablet_copy_client.cc:806] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:33.920271 16278 tablet_copy_client.cc:670] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:33.924588 15847 raft_consensus.cc:1217] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:33.933176 16278 tablet_copy_client.cc:538] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:33.946979 16278 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:33.994602 16278 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:33.995569 16278 tablet_bootstrap.cc:492] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:33.996470 16278 ts_tablet_manager.cc:1403] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.050s	user 0.023s	sys 0.004s
I20260709 13:02:34.001462 16278 raft_consensus.cc:359] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:34.002497 16278 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:34.003108 16278 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:34.005985 16278 ts_tablet_manager.cc:1434] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:02:34.007827 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-0afe5ac407a04216b625837a5bca525f received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:34.008278 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-0afe5ac407a04216b625837a5bca525f on tablet 0afe5ac407a04216b625837a5bca525f with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:34.018754 15237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:34.019443 15237 raft_consensus.cc:606] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:34.037613 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:34.038286 15084 raft_consensus.cc:606] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:34.041399 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:34.041976 15084 raft_consensus.cc:606] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:34.043942 16278 ts_tablet_manager.cc:933] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.046672 15232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:34.047508 15232 raft_consensus.cc:606] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:34.053673 15237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:34.054230 15237 raft_consensus.cc:606] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:34.054702 16278 tablet_copy_client.cc:323] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:34.055401 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:34.056718 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet ec09a7a4d59c4296aa02dde6bd37f81c from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:34.056988 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:34.057265 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet ec09a7a4d59c4296aa02dde6bd37f81c from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-ec09a7a4d59c4296aa02dde6bd37f81c
I20260709 13:02:34.065085 15247 tablet_copy_source_session.cc:215] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:34.069082 16278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec09a7a4d59c4296aa02dde6bd37f81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:34.087196 16278 tablet_copy_client.cc:806] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:34.087685 16278 tablet_copy_client.cc:670] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:34.094894 15847 raft_consensus.cc:1217] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:34.100675 16278 tablet_copy_client.cc:538] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:34.096774 15848 raft_consensus.cc:1217] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:34.113171 16278 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:34.132630 16278 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:34.133133 16262 raft_consensus.cc:993] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:34.133697 16262 raft_consensus.cc:1081] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:34.135481 16278 tablet_bootstrap.cc:492] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:34.136268 16278 ts_tablet_manager.cc:1403] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:02:34.136415 15158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:34.137090 15158 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:34.137419 15158 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.139607 16278 raft_consensus.cc:359] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:34.140527 16278 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:34.141302 16278 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:34.144151 16278 ts_tablet_manager.cc:1434] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:02:34.145996 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ec09a7a4d59c4296aa02dde6bd37f81c received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:34.146508 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ec09a7a4d59c4296aa02dde6bd37f81c on tablet ec09a7a4d59c4296aa02dde6bd37f81c with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:34.149513 15158 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.154917 15158 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.155081 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:34.155328 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:34.155751 15237 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.155956 15084 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.156335 15084 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:34.157115 15084 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.158151 15084 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.163530 15237 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:34.165198 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:34.166074 16134 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.166483 16134 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:34.167193 16134 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.168457 15084 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:34.188076 14982 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.297195 16216 raft_consensus.cc:993] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:34.297767 16288 raft_consensus.cc:1081] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:34.299278 15158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:34.299732 15158 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:34.300065 15158 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.304461 15158 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.306061 15158 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.306845 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:34.307178 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:34.307482 15084 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.307771 15237 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.308128 15237 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:34.308815 15237 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.310120 15237 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.314281 15084 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:34.315675 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:34.315850 15237 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:34.316469 16134 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.316974 16134 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:34.317687 16134 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.327661 14982 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.365074 16288 raft_consensus.cc:1064] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:34.368716 16288 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.376664 15847 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.378134 15084 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.378083 16288 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:34.380311 15158 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.380571 16288 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:34.381922 16286 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:34.412446 16288 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.415392 15087 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.423055 16216 raft_consensus.cc:993] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:34.423791 16267 raft_consensus.cc:1081] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:34.429472 15087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:34.430069 15087 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:34.430429 15087 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.431382 15847 raft_consensus.cc:1217] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:34.437213 15087 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.439990 15087 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.458319 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:34.458973 15158 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.459688 15847 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.462674 14983 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.476475 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:34.477160 15232 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.477478 15232 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:34.478294 15232 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.479416 15232 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.500003 16265 raft_consensus.cc:1064] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:34.502972 16265 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.503551 15160 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.516692 16288 raft_consensus.cc:993] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:34.521739 16246 raft_consensus.cc:1081] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:34.529125 15083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:34.529889 15083 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:34.530290 15083 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.534224 15158 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:34.535295 15159 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:34.543131 15083 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.546911 15235 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.557121 16263 raft_consensus.cc:993] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:34.557736 16263 raft_consensus.cc:1081] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:34.558921 15847 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:34.559830 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:34.560424 15848 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:34.560792 15848 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.561312 16266 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:34.552397 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:34.569016 16262 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.569607 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:34.569893 15087 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:34.570180 15237 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.569620 16262 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:34.570482 15237 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:34.571502 16265 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.571642 15237 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.571717 16262 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.572734 15237 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.565310 15083 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.577059 15846 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:34.568667 15848 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.583673 15229 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:34.584370 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:34.584986 15234 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.585300 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:34.585903 15087 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.586201 15087 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:34.589005 15087 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.589825 16266 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.590238 15087 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.591667 15232 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:34.591566 16267 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.597450 15087 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:34.598821 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:34.600077 16285 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.600564 16285 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:34.601432 16285 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.602116 15087 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:34.604182 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:34.604823 15158 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:34.605825 16240 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.607399 15845 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.616047 15848 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:34.618577 16288 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.620716 15234 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:34.625119 15237 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:34.627755 16267 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.631930 16246 raft_consensus.cc:1064] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:34.635483 15087 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.640606 15084 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.645463 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 06f7ae7c1d614611bb074a65fcc9a959 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:34.651329 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:34.651149 16246 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.653391 16262 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.653980 16262 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:34.654910 16262 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:34.657621 15848 raft_consensus.cc:2955] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.657972 15237 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.664613 15158 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:34.676519 14981 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.677006 14985 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.678650 14982 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.691856 15848 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.693023 15084 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.693918 16245 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:34.694959 16288 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:34.697276 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 06f7ae7c1d614611bb074a65fcc9a959 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:34.701464 15158 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.706396 15237 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:34.708408 16265 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:34.711431 16245 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:34.713871 14983 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.717252 16245 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.727911 15087 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.741014 15848 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.753639 15084 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:34.754997 16317 tablet_replica.cc:333] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:34.755226 16134 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:34.763167 16317 raft_consensus.cc:2243] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:34.764094 16317 raft_consensus.cc:2272] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:34.775575 15160 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.777271 16317 ts_tablet_manager.cc:1916] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:34.783277 15158 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.790983 14982 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.803220 16317 ts_tablet_manager.cc:1929] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:34.804512 16317 log.cc:1199] T 06f7ae7c1d614611bb074a65fcc9a959 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/06f7ae7c1d614611bb074a65fcc9a959
I20260709 13:02:34.808933 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 06f7ae7c1d614611bb074a65fcc9a959 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:34.811055 14984 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.830487 15087 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:34.831772 15848 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:34.834424 16265 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:34.838366 16240 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:34.848443 16266 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.856017 15087 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.865764 15234 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:34.870324 16134 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:34.872295 16267 raft_consensus.cc:1064] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:34.877029 15848 raft_consensus.cc:2955] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:34.875888 16267 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.894325 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0afe5ac407a04216b625837a5bca525f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:34.910022 15158 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.911307 15848 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.912869 15235 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:34.913262 16245 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:34.919461 15084 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:34.920608 16286 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:34.922478 16286 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.923444 15087 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:34.941220 16216 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:34.948217 15848 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:34.954077 15087 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:34.956439 16134 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:34.965636 16216 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.965427 14981 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:35.093886 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:35.126703 16330 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
W20260709 13:02:35.209057 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 177412us
W20260709 13:02:35.245155 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 59692.5us
W20260709 13:02:35.256989 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 191880us
W20260709 13:02:35.257427 16329 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:02:35.282938 15087 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:35.288875 15163 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:35.296300 16285 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:02:35.332388 16341 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:35.360443 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 113928us
W20260709 13:02:35.361421 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 127738us
I20260709 13:02:35.394197 16307 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:35.395021 15848 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.392321 15235 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:35.399049 15234 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:35.408057 15082 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.399049 15229 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:35.408893 15081 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.425812 15847 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.431460 15159 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:35.370110 16216 raft_consensus.cc:2955] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.443040 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 0afe5ac407a04216b625837a5bca525f with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:35.453867 16263 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:35.455394 16347 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:35.456149 16285 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:35.481276 16263 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:35.488777 14983 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:35.510372 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:35.524394 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a5f358b91ee427880ebecaca8118e4f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:35.573747 16351 tablet_replica.cc:333] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:35.577212 16351 raft_consensus.cc:2243] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:35.580250 16351 raft_consensus.cc:2272] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:35.591980 16351 ts_tablet_manager.cc:1916] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:35.600759 14984 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.603112 15160 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.619874 16351 ts_tablet_manager.cc:1929] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:35.620483 16351 log.cc:1199] T 0afe5ac407a04216b625837a5bca525f P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/0afe5ac407a04216b625837a5bca525f
I20260709 13:02:35.631266 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 0afe5ac407a04216b625837a5bca525f (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:35.648715 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a5f358b91ee427880ebecaca8118e4f with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:35.671670 16360 tablet_replica.cc:333] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:35.672977 16360 raft_consensus.cc:2243] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:35.673899 16360 raft_consensus.cc:2272] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:35.688051 16360 ts_tablet_manager.cc:1916] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:35.712178 16360 ts_tablet_manager.cc:1929] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:35.713179 16360 log.cc:1199] T 0a5f358b91ee427880ebecaca8118e4f P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/0a5f358b91ee427880ebecaca8118e4f
I20260709 13:02:35.715464 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 0a5f358b91ee427880ebecaca8118e4f (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:35.715799 15228 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:35.721772 15848 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:35.723519 16344 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:35.724254 15162 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:35.725996 16345 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:35.733414 16344 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.742486 15848 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.742076 15162 raft_consensus.cc:2955] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:35.747522 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec09a7a4d59c4296aa02dde6bd37f81c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:35.752943 14983 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:35.768576 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet ec09a7a4d59c4296aa02dde6bd37f81c with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:35.777040 16361 tablet_replica.cc:333] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:35.778404 16361 raft_consensus.cc:2243] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:35.779609 16361 raft_consensus.cc:2272] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:35.779887 15228 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:35.780603 15228 raft_consensus.cc:606] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:35.785235 16361 ts_tablet_manager.cc:1916] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:35.803128 15162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:35.803743 15162 raft_consensus.cc:606] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:35.805531 16361 ts_tablet_manager.cc:1929] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:35.806054 16361 log.cc:1199] T ec09a7a4d59c4296aa02dde6bd37f81c P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/ec09a7a4d59c4296aa02dde6bd37f81c
I20260709 13:02:35.808140 15160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:35.808246 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet ec09a7a4d59c4296aa02dde6bd37f81c (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:35.808625 15160 raft_consensus.cc:606] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:35.811299 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b358d067cea0428190a484601dea7c69"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:35.811820 15080 raft_consensus.cc:606] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:35.814213 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:35.814785 15080 raft_consensus.cc:606] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:35.816483 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:35.817927 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:35.854920 16346 raft_consensus.cc:993] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:35.855480 16346 raft_consensus.cc:1081] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:35.857131 15080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:35.857692 15080 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:35.858043 15080 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.858141 16353 raft_consensus.cc:993] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:35.858707 16345 raft_consensus.cc:1081] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:35.860116 15162 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:35.860522 15162 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:35.860795 15162 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.863220 15080 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.864908 15080 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:35.865543 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:35.866106 15160 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:35.866197 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:35.866425 15160 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:35.866741 15235 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.867188 15160 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.864854 15162 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.868381 15160 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.870088 15162 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:35.872210 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:35.872736 15234 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:35.873039 15234 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:35.873759 15234 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.874958 15234 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.875120 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:35.875737 15080 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.878957 15235 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:35.880208 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:35.881265 16307 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:35.881734 16307 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:35.882519 16307 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.883112 15234 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:35.883476 15080 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:35.884644 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:35.885578 16348 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:35.886222 16348 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:35.887471 16348 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:35.887343 15160 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:35.897541 16334 raft_consensus.cc:993] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:35.898151 16339 raft_consensus.cc:1081] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:35.899845 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b358d067cea0428190a484601dea7c69"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:35.900437 15848 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:35.900736 15848 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:35.907775 14985 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:35.907208 14983 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:35.913332 15848 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:35.916507 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:35.917104 15162 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:35.918097 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:35.918676 15080 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:35.918932 15080 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:35.919575 15080 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:35.920550 15080 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:35.923941 15162 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:35.926744 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:35.927299 15080 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:35.927773 16343 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:35.928278 16343 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:35.929136 16343 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:35.935364 15848 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:35.946708 14983 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.160691 16348 raft_consensus.cc:993] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:36.161147 16348 raft_consensus.cc:1081] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:36.162367 15080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:36.162829 15080 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:36.163061 15080 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.167467 15080 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:36.168984 15080 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:36.169786 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:36.169863 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:36.170289 15160 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:36.170393 15235 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.170667 15160 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:36.171427 15160 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:36.172291 15160 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.175712 15235 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:36.176810 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:36.177552 16304 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:36.177732 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:36.177955 16304 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:36.178936 16304 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:36.192723 14983 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:36.276698 15160 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.277909 16334 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.288465 15235 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.289819 16334 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.310091 15160 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:36.311888 15235 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.313421 16343 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:36.318104 16346 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.335810 15080 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.337723 15082 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:36.339176 16343 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:36.339216 16346 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:36.346966 16304 raft_consensus.cc:993] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:36.347527 16307 raft_consensus.cc:1081] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:36.349216 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:36.349740 15848 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:36.350052 15848 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.361588 15848 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:36.365499 15848 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:36.365542 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:36.366289 15080 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:36.366632 15080 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:36.366485 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:36.367313 15235 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.367460 15080 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:36.378135 15235 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:36.379688 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:36.381045 16343 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:36.381567 16343 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:36.382499 16343 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:36.382983 15080 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.395340 15080 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:36.404701 14985 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.708660 15159 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.710103 16334 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.714520 15235 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:36.716008 16334 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.808451 15234 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:36.809888 16285 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:36.819013 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:36.823208 15082 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:36.824558 16343 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:36.876400 15080 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:36.885092 15234 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:36.885457 15052 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d50a21b24380415781c0e63c5d613aff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:36.886977 16304 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.893240 15159 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:36.894694 16304 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.898461 16307 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.900156 15234 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.922593 15080 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:36.927080 15159 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.934238 14983 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:36.948184 15052 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 661df855a98f493ca654519f6be2164f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:36.951988 15234 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:36.951974 15162 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:36.953981 16307 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.956920 16304 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.971688 16304 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.976552 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:36.978600 15234 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.980693 15159 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:36.989240 15160 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:36.992863 14981 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:37.023026 15127 consensus_peers.cc:597] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e0c065a7fec4f5c88b4bc85b3508231. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:37.027227 15080 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:37.028008 15234 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:37.030783 16348 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:37.036661 16336 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:37.054546 16336 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.056571 15080 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.057955 15234 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.062570 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:37.063134 15235 raft_consensus.cc:606] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.069294 15160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:37.069844 15160 raft_consensus.cc:606] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.068454 15087 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.073359 14981 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.078347 15235 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:37.079345 15160 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:37.078464 15052 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73ae83bb85154bb08a2889f0f9f99cd8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:37.080639 15087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:37.080565 16304 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:37.082751 15087 raft_consensus.cc:606] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:37.085996 16307 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:37.104503 15087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:37.105468 15087 raft_consensus.cc:606] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.108495 15234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a12ada8057b141518113efeb519c6514"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:37.111590 16307 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.113282 15234 raft_consensus.cc:606] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.115269 15159 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.116417 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:37.117899 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:37.118660 15235 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.135977 14981 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:37.299532 16346 raft_consensus.cc:993] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.300036 16346 raft_consensus.cc:1081] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.302224 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59152
I20260709 13:02:37.302742 15848 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.303062 15848 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.304734 16344 raft_consensus.cc:993] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.305330 16353 raft_consensus.cc:1081] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.307036 15847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59122
I20260709 13:02:37.307518 15847 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.307832 15847 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.316720 15847 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.318800 15847 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:37.321311 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:37.321856 15235 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.322139 15235 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:37.322810 15235 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.323807 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:37.323982 15235 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.324128 15848 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.326447 15848 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:37.324416 15087 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.327481 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:37.328073 15159 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.328521 15159 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:37.329210 15159 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.330324 15159 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.336966 15087 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.338567 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:37.339149 15234 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.339118 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:37.345116 16343 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:37.345639 16343 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:37.346481 16343 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.357508 15159 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.393744 15234 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.395147 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:37.396670 16404 ts_tablet_manager.cc:933] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:37.397089 16382 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:37.398644 16382 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:37.399616 16382 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.409624 16404 tablet_copy_client.cc:323] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:37.410837 16307 raft_consensus.cc:993] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.352499 15235 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.412418 16307 raft_consensus.cc:1081] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.412958 16401 ts_tablet_manager.cc:933] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:37.414098 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:37.414770 15848 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.415233 15848 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.417266 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet d50a21b24380415781c0e63c5d613aff from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:37.417829 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet d50a21b24380415781c0e63c5d613aff from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-d50a21b24380415781c0e63c5d613aff
I20260709 13:02:37.423188 16401 tablet_copy_client.cc:323] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:37.420100 14985 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.425509 15097 tablet_copy_source_session.cc:215] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:37.425846 15848 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.427922 15848 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:37.429840 16404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50a21b24380415781c0e63c5d613aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:37.430576 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:37.430701 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 661df855a98f493ca654519f6be2164f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:37.431149 15087 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.431280 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 661df855a98f493ca654519f6be2164f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-661df855a98f493ca654519f6be2164f
I20260709 13:02:37.431517 15087 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:37.432260 15087 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.435678 15087 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.437961 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:37.438512 15159 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.442214 15087 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.443224 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:37.444129 16343 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:37.444597 16343 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:37.445689 16404 tablet_copy_client.cc:806] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:37.445426 16343 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.446228 16404 tablet_copy_client.cc:670] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:37.452135 15097 tablet_copy_source_session.cc:215] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:37.457230 16404 tablet_copy_client.cc:538] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:37.465844 16401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 661df855a98f493ca654519f6be2164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:37.465724 15159 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.483358 16401 tablet_copy_client.cc:806] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:37.484055 16401 tablet_copy_client.cc:670] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:37.487995 16404 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:37.490131 16401 tablet_copy_client.cc:538] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:37.501739 16401 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:37.502068 14985 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.505791 14985 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.512633 16409 ts_tablet_manager.cc:933] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:37.523878 16409 tablet_copy_client.cc:323] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:37.525787 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet 73ae83bb85154bb08a2889f0f9f99cd8 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:37.526320 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet 73ae83bb85154bb08a2889f0f9f99cd8 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-73ae83bb85154bb08a2889f0f9f99cd8
I20260709 13:02:37.535632 16344 raft_consensus.cc:993] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.540992 16344 raft_consensus.cc:1081] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:37.541771 15097 tablet_copy_source_session.cc:215] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:37.542801 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a12ada8057b141518113efeb519c6514"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59122
I20260709 13:02:37.543395 15848 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.543725 15848 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.548955 16409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ae83bb85154bb08a2889f0f9f99cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:37.550770 16404 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:37.551806 16404 tablet_bootstrap.cc:492] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:37.552578 16404 ts_tablet_manager.cc:1403] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.065s	user 0.026s	sys 0.000s
I20260709 13:02:37.555670 15848 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.559067 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:37.559705 15087 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.565773 16401 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:37.566363 15087 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.566675 16401 tablet_bootstrap.cc:492] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:37.555938 16404 raft_consensus.cc:359] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.567705 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:37.567840 16404 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:37.568567 16343 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:37.568957 16401 ts_tablet_manager.cc:1403] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.019s
I20260709 13:02:37.569988 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:37.578284 15848 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:37.582307 16401 raft_consensus.cc:359] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.583205 16401 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:37.583874 16401 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.579814 15235 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.586652 15235 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:37.588613 15235 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.568763 16404 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.590314 15235 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.599047 16404 ts_tablet_manager.cc:1434] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:02:37.600952 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-d50a21b24380415781c0e63c5d613aff received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:37.601380 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-d50a21b24380415781c0e63c5d613aff on tablet d50a21b24380415781c0e63c5d613aff with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.603473 16343 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:37.611248 16401 ts_tablet_manager.cc:1434] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.003s
I20260709 13:02:37.611712 15235 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:37.612177 16343 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:37.613956 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-661df855a98f493ca654519f6be2164f received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:37.614390 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-661df855a98f493ca654519f6be2164f on tablet 661df855a98f493ca654519f6be2164f with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.618345 16307 raft_consensus.cc:993] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:37.618803 16307 raft_consensus.cc:1081] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:37.621505 15159 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:37.622007 15159 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.622352 15159 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.633344 16409 tablet_copy_client.cc:806] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:37.637449 16409 tablet_copy_client.cc:670] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:37.634428 16401 ts_tablet_manager.cc:933] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:37.636397 15159 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:37.639993 16401 tablet_copy_client.cc:323] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:37.640321 15159 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:37.640542 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:37.641186 15087 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.641286 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:37.641548 15087 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:37.641870 15235 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.642091 14985 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.649178 15235 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:37.651453 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:37.653431 16336 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:37.653661 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 3e0c065a7fec4f5c88b4bc85b3508231 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:37.654824 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 3e0c065a7fec4f5c88b4bc85b3508231 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-3e0c065a7fec4f5c88b4bc85b3508231
I20260709 13:02:37.642328 15087 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:37.657107 15087 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.659034 16409 tablet_copy_client.cc:538] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:37.665621 15087 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:37.666636 16336 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:37.667956 16336 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:37.683687 15173 tablet_copy_source_session.cc:215] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:37.690013 16401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0c065a7fec4f5c88b4bc85b3508231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:37.681345 16409 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:37.710330 16401 tablet_copy_client.cc:806] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:37.711218 16401 tablet_copy_client.cc:670] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:37.711905 14985 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.715399 16401 tablet_copy_client.cc:538] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:37.721063 16409 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:37.722121 16409 tablet_bootstrap.cc:492] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:37.722857 16409 ts_tablet_manager.cc:1403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.007s
I20260709 13:02:37.725124 16401 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:37.726253 16409 raft_consensus.cc:359] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.727145 16409 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:37.727792 16409 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.730576 16409 ts_tablet_manager.cc:1434] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:02:37.732812 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-73ae83bb85154bb08a2889f0f9f99cd8 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:37.733251 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-73ae83bb85154bb08a2889f0f9f99cd8 on tablet 73ae83bb85154bb08a2889f0f9f99cd8 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.754779 16401 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:37.755744 16401 tablet_bootstrap.cc:492] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:37.756434 16401 ts_tablet_manager.cc:1403] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.008s
I20260709 13:02:37.759469 16401 raft_consensus.cc:359] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.760301 16401 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:37.760933 16401 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:37.763372 16401 ts_tablet_manager.cc:1434] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:37.765618 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-3e0c065a7fec4f5c88b4bc85b3508231 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:37.766054 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-3e0c065a7fec4f5c88b4bc85b3508231 on tablet 3e0c065a7fec4f5c88b4bc85b3508231 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:37.783586 15848 raft_consensus.cc:1217] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:37.804828 15234 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.806555 16343 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.822677 15087 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.824445 16343 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:37.843506 15160 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.845146 16285 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.845846 15082 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.847167 16343 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.873171 15159 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.877784 15235 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:37.876595 16285 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.880333 16343 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.931598 15848 raft_consensus.cc:1217] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:02:37.971751 16423 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:38.017906 15087 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:38.019141 16430 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:38.032418 15229 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:38.035176 16429 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:38.036227 15848 raft_consensus.cc:1217] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:38.118772 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:38.149417 15229 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:38.151115 16346 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:38.225219 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:38.248859 15087 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:38.262458 16348 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:38.271577 15848 raft_consensus.cc:1217] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:38.306913 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:38.307512 15235 raft_consensus.cc:606] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:38.313100 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:38.313663 15235 raft_consensus.cc:606] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:38.315237 16304 raft_consensus.cc:1064] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:38.318246 15162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:38.317708 16304 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:38.318804 15162 raft_consensus.cc:606] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:38.322634 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:38.323410 15235 raft_consensus.cc:606] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:38.325264 15848 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.325853 15235 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.326764 15162 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.328972 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:38.329521 15235 raft_consensus.cc:606] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:38.326725 16304 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:38.332255 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:38.335386 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:38.344769 16334 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.347601 16307 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:38.372783 16378 raft_consensus.cc:993] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.373355 16428 raft_consensus.cc:1081] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.376210 15080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:38.376940 15080 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:38.377283 15080 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.378999 16307 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.380762 15229 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.391687 15847 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.400573 14985 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:38.404461 15162 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.410151 15080 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.419305 15080 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:38.421082 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:38.421733 15159 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.425647 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:38.426131 15229 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:38.426304 15229 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:38.426863 15229 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.427778 15229 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.433753 15229 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:38.434909 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:38.435870 16307 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:38.436524 16307 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:38.437561 16307 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.453594 15080 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:38.460776 15229 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:38.462347 16304 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:38.468974 15847 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:38.470983 16334 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:38.475203 16334 raft_consensus.cc:1064] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:38.478232 16334 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:38.479882 16425 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.481132 15229 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.478720 15159 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
W20260709 13:02:38.531204 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47176.9us
I20260709 13:02:38.524278 14983 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.537302 16449 raft_consensus.cc:1064] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:38.544968 16449 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:38.637902 15229 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:02:38.649418 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 115621us
I20260709 13:02:38.639302 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d50a21b24380415781c0e63c5d613aff with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:02:38.692276 14972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 65821.2us
I20260709 13:02:38.758293 15848 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.758785 16464 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:02:38.768091 16425 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:38.771586 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet d50a21b24380415781c0e63c5d613aff with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:38.773005 16462 raft_consensus.cc:993] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:38.774263 16462 raft_consensus.cc:1081] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:38.774734 14985 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:38.783886 16461 raft_consensus.cc:993] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:38.784478 16461 raft_consensus.cc:1081] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:38.803869 15159 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.832618 15160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:38.832823 15162 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:38.833267 15160 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:38.833709 15160 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.833575 15162 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:38.834934 15162 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.846999 16304 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:38.853062 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 91767.1us
I20260709 13:02:38.858846 15848 raft_consensus.cc:2955] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.863803 15162 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.904569 15162 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:38.876463 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55931.4us
I20260709 13:02:38.878057 16336 raft_consensus.cc:1064] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:38.911145 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:38.913772 15235 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:38.914076 15235 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:38.915436 16336 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:38.925380 15158 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.915422 15235 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.931689 15235 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.940569 15229 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.944437 16304 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:38.950440 16436 raft_consensus.cc:993] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.950984 16436 raft_consensus.cc:1081] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.951298 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:38.951958 15087 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.953473 16304 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.959687 15087 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:38.968164 16452 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:38.896508 15160 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.906991 15845 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.873544 16467 tablet_replica.cc:333] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:38.971782 16467 raft_consensus.cc:2243] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.972494 16467 raft_consensus.cc:2272] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.910750 16468 raft_consensus.cc:993] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.973457 16468 raft_consensus.cc:1081] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:38.977033 15160 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:38.969611 15229 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.977267 15845 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:38.982028 16459 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.982097 15847 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:38.982446 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:38.984685 15228 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:38.985013 16459 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.985296 15228 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:38.986428 15228 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:38.987907 15228 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:38.993259 16467 ts_tablet_manager.cc:1916] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:39.000617 16449 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:39.004300 15162 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.007586 15237 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.014882 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:39.018633 15082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:39.019274 15082 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.019740 15082 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.022322 15087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:39.023252 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:39.023806 15084 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.023842 15083 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:39.024194 16458 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.024677 16458 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.025517 16458 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.026937 15082 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.029011 15087 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.029538 15087 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.029870 14985 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.031286 15163 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.029793 15228 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:39.032796 15082 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.033735 15126 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:39.034597 16436 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.035140 16436 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.043026 16436 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.034889 15084 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:39.050433 15235 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:39.041900 15087 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.049880 14981 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.053807 15087 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.066757 15237 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.069972 16425 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.071159 16436 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:39.074481 16464 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.081430 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:39.082098 15163 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.083204 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:39.083747 15159 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:39.084028 15159 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.084697 15159 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.085796 15159 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.090179 15163 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:39.091846 16467 ts_tablet_manager.cc:1929] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:39.092303 16467 log.cc:1199] T d50a21b24380415781c0e63c5d613aff P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/d50a21b24380415781c0e63c5d613aff
I20260709 13:02:39.094285 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet d50a21b24380415781c0e63c5d613aff (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
W20260709 13:02:39.096463 16477 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:39.098562 15159 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:39.100265 15234 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.110350 16458 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.120798 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.121445 15235 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:39.122371 15229 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.121753 15235 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:39.141038 14983 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.143566 15845 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.150005 14981 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.155264 15235 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.156438 15848 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.157451 15235 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.162019 15054 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:39.164500 15235 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:39.171332 16304 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.172048 15054 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:39.173956 16304 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:39.174378 16304 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.175102 16304 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.172996 16449 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.178974 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.190706 15237 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.177145 14983 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.206637 16449 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.207816 16449 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.216382 15158 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.217428 15160 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.217631 15087 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.226552 15848 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.228255 16458 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.233704 15229 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.238803 15082 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.241128 16346 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.244633 15848 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.253302 16336 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.262035 15847 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.265043 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e0c065a7fec4f5c88b4bc85b3508231 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:39.266047 15163 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.267017 16304 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.267828 16334 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.291020 15234 raft_consensus.cc:2955] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.293972 16464 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.310957 15237 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:39.313942 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 661df855a98f493ca654519f6be2164f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:39.320503 14985 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.335980 15847 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.340490 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:39.357407 15163 raft_consensus.cc:2955] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.375401 14983 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.377219 14981 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.383296 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e0c065a7fec4f5c88b4bc85b3508231 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:39.377871 14983 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.398092 16488 tablet_replica.cc:333] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:39.401818 16488 raft_consensus.cc:2243] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.402674 16488 raft_consensus.cc:2272] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.410207 16488 ts_tablet_manager.cc:1916] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:39.413187 15084 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.420163 15847 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.421934 16334 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.429306 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 661df855a98f493ca654519f6be2164f with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:39.430729 15235 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:39.433082 16488 ts_tablet_manager.cc:1929] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:39.433707 16488 log.cc:1199] T 3e0c065a7fec4f5c88b4bc85b3508231 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/3e0c065a7fec4f5c88b4bc85b3508231
I20260709 13:02:39.434950 16489 tablet_replica.cc:333] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:39.436312 16449 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:39.438884 16489 raft_consensus.cc:2243] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.440210 16489 raft_consensus.cc:2272] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.439392 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 3e0c065a7fec4f5c88b4bc85b3508231 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:39.441504 16449 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.446933 15234 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.447696 15847 raft_consensus.cc:2955] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:39.449689 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:39.444763 15235 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.451785 16346 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.453738 16489 ts_tablet_manager.cc:1916] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:39.456012 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73ae83bb85154bb08a2889f0f9f99cd8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:39.466281 14981 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.478961 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 73ae83bb85154bb08a2889f0f9f99cd8 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:39.479946 16467 tablet_replica.cc:333] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:39.480927 16467 raft_consensus.cc:2243] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.481609 16467 raft_consensus.cc:2272] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.478154 16489 ts_tablet_manager.cc:1929] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:39.486116 16489 log.cc:1199] T 661df855a98f493ca654519f6be2164f P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/661df855a98f493ca654519f6be2164f
I20260709 13:02:39.488036 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 661df855a98f493ca654519f6be2164f (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:39.490392 16467 ts_tablet_manager.cc:1916] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:39.499742 15084 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.506524 16436 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.507634 16467 ts_tablet_manager.cc:1929] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:39.509366 16467 log.cc:1199] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/73ae83bb85154bb08a2889f0f9f99cd8
I20260709 13:02:39.511073 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 73ae83bb85154bb08a2889f0f9f99cd8 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:39.521229 15087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:39.521831 15087 raft_consensus.cc:606] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:39.527913 15234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:39.528482 15234 raft_consensus.cc:606] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:39.531806 15084 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.533330 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:39.533625 16346 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.534072 15163 raft_consensus.cc:606] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:39.537364 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:39.537981 15082 raft_consensus.cc:606] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:39.546934 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:39.548327 15084 raft_consensus.cc:606] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:39.552264 15229 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.555143 16458 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.557269 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:39.557794 15163 raft_consensus.cc:606] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:39.559942 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:39.562315 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:39.663573 15229 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:39.665009 16449 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:39.676427 15158 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:39.677695 16334 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:39.685330 16334 raft_consensus.cc:993] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:39.685781 16449 raft_consensus.cc:1081] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:39.687384 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:39.687922 15163 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.688985 15163 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.696552 15163 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.699265 15163 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.700304 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.700639 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:39.701229 15087 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:39.701545 15087 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.702270 15087 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.702636 15229 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.703408 15087 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.708633 15229 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:39.709793 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:39.710114 15087 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:39.710409 16346 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:39.710894 16346 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.711644 16346 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.721424 14985 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.728663 16449 raft_consensus.cc:993] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:39.729336 16449 raft_consensus.cc:1081] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:39.731038 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:39.731546 15848 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.731860 15848 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.736487 15848 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.738214 15848 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:39.739187 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:39.738899 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:39.739722 15163 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.739996 15087 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:39.740311 15087 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.741073 15087 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.742269 15087 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.746452 15163 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:39.748659 15087 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:39.748755 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:39.749680 16429 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.750180 16429 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:39.750385 15229 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.751055 16429 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.752082 16449 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.762567 14985 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.769546 15163 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:39.772750 16448 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:39.783686 16449 raft_consensus.cc:993] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:39.784168 16449 raft_consensus.cc:1081] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:39.786254 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:39.786777 15848 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.787129 15848 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.795615 15848 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.799705 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.800386 15234 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.802274 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:39.802878 15087 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:39.803148 15087 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.804142 15848 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.804032 15087 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.805192 15087 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.808068 15234 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:39.809195 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:39.810038 16429 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.810250 15087 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:39.810582 16429 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:39.811372 16429 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:39.820936 14985 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.825649 16436 raft_consensus.cc:993] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:39.826213 16457 raft_consensus.cc:1081] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:39.827777 15087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:39.828227 15087 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.828514 15087 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.833309 15087 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.835068 15087 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.836011 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:39.836603 15163 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:39.836949 15163 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.837690 15163 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.838757 15163 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.841207 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.841739 15234 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.846434 15163 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:39.848637 15054 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:39.849577 16448 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:39.850101 16448 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.850919 16448 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.857426 15234 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:39.866151 14985 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.905491 16458 raft_consensus.cc:993] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:39.905987 16457 raft_consensus.cc:1081] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:39.907431 15087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:39.907905 15087 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.908258 15087 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.912469 15087 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.913837 15087 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.914674 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:39.914685 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.915335 15163 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:39.915431 15234 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.915635 15163 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:39.916555 15163 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.917621 15163 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.921244 15234 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:39.922312 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:39.922624 15163 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:39.923097 16448 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:39.923676 16448 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:39.924503 16448 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.934396 14985 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.982144 16468 raft_consensus.cc:993] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:39.982702 16353 raft_consensus.cc:1081] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:39.984149 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:39.984611 15163 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.984972 15163 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.989833 15163 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.991425 15163 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:39.992658 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:39.992626 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:39.993428 15234 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:39.993561 15087 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.993763 15234 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:39.994640 15234 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:39.995478 15234 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.998381 15087 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:39.999531 15128 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:40.000180 16436 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:40.000661 16436 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:40.000815 15234 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:40.001490 16436 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:40.012259 14985 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:40.173162 15084 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:40.174461 16382 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:40.183578 15163 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:40.185104 16382 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:40.192559 15234 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:40.193794 16457 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:40.203871 15084 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:40.205183 16457 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:40.260552 15084 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:40.261776 16382 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:40.268827 15234 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:40.270130 15163 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:40.272022 16429 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:40.273586 16304 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:40.297142 15234 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:40.300453 16449 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:40.338891 15229 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:40.340111 16448 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.353421 15158 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:40.354737 16510 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.544510 15229 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:40.545848 16436 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:40.551790 15034 auto_rebalancer.cc:629] Failed to schedule move for tablet 211046e319804173a9b8954112b8d044: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:40.555136 15084 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:40.556448 16458 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.560348 15158 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:40.563211 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:40.575873 15229 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:40.576273 15084 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:40.577291 16346 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.581554 16346 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:40.584246 15127 consensus_peers.cc:597] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c84610e2d65945cab0886909d43d5e9e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:40.590349 16436 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.592099 15084 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.593947 15229 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.613384 15158 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:40.613607 14984 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.630777 15229 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:02:40.637238 15127 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15cd34bca3be499d87220d6e1b2a763f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:40.638732 16436 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:40.646508 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:40.653020 15084 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:40.654376 16436 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.656442 16346 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:40.656013 15235 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.672149 15234 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:40.682358 15087 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:40.702394 15201 consensus_peers.cc:597] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d62d65616394d059c479b07e82eccb9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:40.704649 15084 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:40.704671 15163 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:40.707144 16468 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.718427 16353 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:40.754606 15082 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.757189 14985 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.749495 16530 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:40.765457 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47492.8us
I20260709 13:02:40.779142 15163 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.801393 14983 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:40.809185 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:40.809824 15163 raft_consensus.cc:606] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:40.814513 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:40.817493 15082 raft_consensus.cc:606] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:40.820223 15234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:40.820767 15234 raft_consensus.cc:606] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:40.823700 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:40.824542 15163 raft_consensus.cc:606] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:40.825812 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:02:40.827036 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:40.934800 16526 raft_consensus.cc:993] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:40.935340 16526 raft_consensus.cc:1081] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:40.942778 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:40.943463 15163 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:40.943869 15163 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:40.951133 15163 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:40.953598 15163 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:40.956614 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:40.957302 15082 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:40.957345 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:40.957998 15234 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:40.958333 15234 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:40.959237 15234 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:40.960019 16436 raft_consensus.cc:993] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:40.960477 15234 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:40.967991 15082 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:40.969319 15234 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:40.960501 16436 raft_consensus.cc:1081] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:40.974347 15848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59152
I20260709 13:02:40.974951 15848 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:40.975345 15848 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:40.981175 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:40.983076 16346 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:40.983606 16346 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:40.986392 16436 raft_consensus.cc:993] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:40.987044 16436 raft_consensus.cc:1081] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:40.987744 15848 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:40.988773 16346 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:40.990401 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:40.990931 15158 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:40.991233 15158 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:40.991879 15158 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:40.992966 15158 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:40.994028 15082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:40.996928 15082 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:40.997298 15082 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:41.006294 15848 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:41.007249 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:41.007001 16532 ts_tablet_manager.cc:933] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:41.008405 15235 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:41.011713 15158 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:41.011576 14985 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.013152 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:41.014840 15235 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:41.015172 16382 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:41.021914 15082 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.026638 15082 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:41.026525 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:41.027208 15158 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:41.027554 15158 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:41.027364 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:41.028331 15848 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:41.029049 16382 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:41.028735 15158 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.030496 15158 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:41.030085 16382 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.037030 16532 tablet_copy_client.cc:323] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:41.040376 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet c84610e2d65945cab0886909d43d5e9e from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:41.041056 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet c84610e2d65945cab0886909d43d5e9e from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-c84610e2d65945cab0886909d43d5e9e
I20260709 13:02:41.041872 15848 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:41.043759 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:41.044906 16511 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:41.045423 16511 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:41.046383 16511 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.051880 15173 tablet_copy_source_session.cc:215] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:41.058362 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84610e2d65945cab0886909d43d5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:41.058983 15158 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:41.071189 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.072335 14983 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.086432 16532 tablet_copy_client.cc:806] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:41.087126 16532 tablet_copy_client.cc:670] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:41.093982 16532 tablet_copy_client.cc:538] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:41.103395 16532 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:41.121971 16532 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:41.122764 16532 tablet_bootstrap.cc:492] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:41.123404 16532 ts_tablet_manager.cc:1403] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:02:41.126402 16532 raft_consensus.cc:359] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.127238 16532 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:41.127841 16532 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.130231 16532 ts_tablet_manager.cc:1434] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:02:41.133549 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-c84610e2d65945cab0886909d43d5e9e received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:41.134060 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-c84610e2d65945cab0886909d43d5e9e on tablet c84610e2d65945cab0886909d43d5e9e with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:41.155980 16304 raft_consensus.cc:993] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:41.156474 16511 raft_consensus.cc:1081] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:41.158344 15235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:02:41.158980 15235 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:41.159346 15235 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:41.165887 15235 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:41.167761 15235 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:41.168778 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:41.168915 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:41.169409 15163 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:41.169566 15082 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:41.169960 15082 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:41.170574 15082 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:41.171501 15082 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:41.175778 15163 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:02:41.177196 15203 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:41.177893 16353 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:41.178328 16353 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:41.179013 16353 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:41.177614 15082 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:02:41.192528 14981 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 3, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.234530 16532 ts_tablet_manager.cc:933] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:41.236317 16532 tablet_copy_client.cc:323] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:41.239038 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 4d62d65616394d059c479b07e82eccb9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:41.239600 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 4d62d65616394d059c479b07e82eccb9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-4d62d65616394d059c479b07e82eccb9
I20260709 13:02:41.254768 16544 ts_tablet_manager.cc:933] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:41.261070 16544 tablet_copy_client.cc:323] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:41.261293 15247 tablet_copy_source_session.cc:215] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:41.264569 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d62d65616394d059c479b07e82eccb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:41.270855 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 15cd34bca3be499d87220d6e1b2a763f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:41.271580 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 15cd34bca3be499d87220d6e1b2a763f from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-15cd34bca3be499d87220d6e1b2a763f
I20260709 13:02:41.282584 15173 tablet_copy_source_session.cc:215] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:41.285533 16544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cd34bca3be499d87220d6e1b2a763f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:41.287678 16532 tablet_copy_client.cc:806] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:41.288275 16532 tablet_copy_client.cc:670] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:41.293831 16532 tablet_copy_client.cc:538] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:41.306882 16532 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:41.323817 16544 tablet_copy_client.cc:806] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:41.324467 16544 tablet_copy_client.cc:670] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:41.334959 16544 tablet_copy_client.cc:538] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:41.337831 16532 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:41.338822 16532 tablet_bootstrap.cc:492] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:41.339521 16532 ts_tablet_manager.cc:1403] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.004s
I20260709 13:02:41.342530 16532 raft_consensus.cc:359] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.343381 16532 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:41.344043 16532 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.346540 16532 ts_tablet_manager.cc:1434] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:02:41.349270 15247 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-4d62d65616394d059c479b07e82eccb9 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:41.349718 15247 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-4d62d65616394d059c479b07e82eccb9 on tablet 4d62d65616394d059c479b07e82eccb9 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:41.353781 16544 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:41.379980 16544 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:41.380759 16544 tablet_bootstrap.cc:492] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:41.381487 16544 ts_tablet_manager.cc:1403] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.005s
I20260709 13:02:41.384096 16544 raft_consensus.cc:359] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.385032 16544 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:41.385645 16544 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:41.387526 16544 ts_tablet_manager.cc:1434] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:02:41.389766 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-15cd34bca3be499d87220d6e1b2a763f received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:41.390169 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-15cd34bca3be499d87220d6e1b2a763f on tablet 15cd34bca3be499d87220d6e1b2a763f with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:41.418828 15848 raft_consensus.cc:1217] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:41.425721 15163 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.427001 16382 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:41.441371 15235 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.444370 16382 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:41.449359 15158 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.458384 16543 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:41.481362 15082 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:41.482832 16533 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:41.523976 15848 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.525609 16511 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:41.563818 15235 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:41.566831 16533 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:41.674182 15848 raft_consensus.cc:1217] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:41.729233 15848 raft_consensus.cc:1217] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:41.765038 15082 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:41.767110 16530 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:41.794798 15160 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:41.796736 16468 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:41.827912 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:41.848369 16346 raft_consensus.cc:1064] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:41.852494 16346 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.871928 15848 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:41.873502 16558 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:41.895978 15234 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:41.896148 15082 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:41.898921 16346 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:41.900825 16458 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:41.912997 16458 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:41.923123 15082 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:41.928309 15234 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:41.944007 15848 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:41.950470 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:41.947235 14983 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:41.978490 15160 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:41.984262 15848 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:41.988955 16458 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:41.996016 15234 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:41.998029 16458 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:42.004180 16346 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.007714 15848 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.011968 15234 raft_consensus.cc:2955] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.031195 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c84610e2d65945cab0886909d43d5e9e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:42.039031 15160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:42.039659 15160 raft_consensus.cc:606] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:42.040470 14984 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.043136 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:42.043778 15082 raft_consensus.cc:606] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:42.047127 15234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de505aa1e895447486f53629c5269998"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:42.048158 15234 raft_consensus.cc:606] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:42.050179 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:42.052009 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:42.054236 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet c84610e2d65945cab0886909d43d5e9e with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:42.074064 16568 tablet_replica.cc:333] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:42.074966 16568 raft_consensus.cc:2243] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:42.075846 16568 raft_consensus.cc:2272] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:42.080117 16568 ts_tablet_manager.cc:1916] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:42.086284 16526 raft_consensus.cc:1064] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:42.089407 16526 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:42.096473 16568 ts_tablet_manager.cc:1929] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:42.097568 15848 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:42.097700 16568 log.cc:1199] T c84610e2d65945cab0886909d43d5e9e P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/c84610e2d65945cab0886909d43d5e9e
I20260709 13:02:42.099905 15160 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:42.110263 15082 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:42.103224 16468 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:42.115293 16468 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:42.117316 16468 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:42.127110 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet c84610e2d65945cab0886909d43d5e9e (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:42.141324 16530 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.142560 15080 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.143675 15847 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.166982 16558 raft_consensus.cc:1064] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:42.171789 16558 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:42.178238 15163 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.188303 14984 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.207393 15080 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:42.195547 15234 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:42.226042 16558 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:42.194169 15847 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:42.230222 16346 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:42.246548 16346 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:42.250083 16458 raft_consensus.cc:993] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:42.250581 16458 raft_consensus.cc:1081] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:42.253527 15080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:42.260210 15080 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.260789 15080 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:42.261569 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 92492.5us
I20260709 13:02:42.264590 15234 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:42.273528 15160 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:42.275089 16353 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:42.275767 15847 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:42.287103 16533 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.296868 15848 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.299531 15082 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.301504 16353 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.312610 16468 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:42.306998 15160 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.315183 15234 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.323397 15080 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.325886 15080 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:42.330217 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:42.330775 15163 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:42.331064 15163 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:42.331774 15163 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.332897 15163 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.337538 14981 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.342338 15848 raft_consensus.cc:2955] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.345206 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:42.351485 15234 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.362250 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4d62d65616394d059c479b07e82eccb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:42.357321 15163 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:42.372442 14981 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.373909 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:42.374909 16304 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:42.378521 15158 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:42.374485 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48378.1us
I20260709 13:02:42.381309 16304 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:42.382262 16304 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.386020 16510 raft_consensus.cc:993] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:42.386546 16510 raft_consensus.cc:1081] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:42.389693 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d62d65616394d059c479b07e82eccb9 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:42.390667 16568 tablet_replica.cc:333] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:42.391666 16568 raft_consensus.cc:2243] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:42.392409 16568 raft_consensus.cc:2272] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:42.397728 16568 ts_tablet_manager.cc:1916] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:42.410739 15848 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:42.412855 16458 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:02:42.435932 16346 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.448976 15848 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.463500 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15cd34bca3be499d87220d6e1b2a763f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:42.481549 16353 raft_consensus.cc:993] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:42.482241 16353 raft_consensus.cc:1081] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:42.481549 15234 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:02:42.483879 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60941.3us
I20260709 13:02:42.483560 14984 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.491998 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de505aa1e895447486f53629c5269998"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:42.492553 15163 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.492924 15163 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:42.499176 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 117596us
I20260709 13:02:42.500862 15234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:02:42.501391 15234 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.501754 15234 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.503913 15082 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:42.505290 16346 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:42.506371 14981 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.503578 15163 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.511104 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 15cd34bca3be499d87220d6e1b2a763f with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:42.511245 15163 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:42.512372 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:42.512984 15235 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.513312 15235 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:42.514076 15235 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.515285 15235 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.512626 15234 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.519426 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:42.520010 15160 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.523784 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:42.524361 15084 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:42.524454 16568 ts_tablet_manager.cc:1929] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:42.524724 15084 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:42.525003 16568 log.cc:1199] T 4d62d65616394d059c479b07e82eccb9 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/4d62d65616394d059c479b07e82eccb9
I20260709 13:02:42.525753 15235 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:42.525610 15084 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.526695 15084 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.526778 15234 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:42.527541 15126 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:42.528292 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 4d62d65616394d059c479b07e82eccb9 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:42.531754 16346 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.537668 16346 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:42.537771 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:42.543213 15082 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.542914 16346 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.540618 15080 raft_consensus.cc:2955] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:42.549806 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:42.542908 16583 tablet_replica.cc:333] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:42.553113 16583 raft_consensus.cc:2243] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:42.555064 16583 raft_consensus.cc:2272] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:42.541183 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:02:42.551687 15084 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:02:42.558060 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:42.558866 16573 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:42.559399 16573 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:42.560679 16583 ts_tablet_manager.cc:1916] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:42.560305 16573 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:42.564210 14984 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.577131 16583 ts_tablet_manager.cc:1929] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:42.577687 16583 log.cc:1199] T 15cd34bca3be499d87220d6e1b2a763f P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/15cd34bca3be499d87220d6e1b2a763f
I20260709 13:02:42.580240 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 15cd34bca3be499d87220d6e1b2a763f (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:42.583668 14984 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 3, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.842521 15235 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:42.844169 16510 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:42.857275 15160 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:42.859447 16510 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:42.985692 15234 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.987087 16346 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.001546 15084 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.004163 16533 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.052908 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:43.137094 15160 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:43.138419 16573 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:43.150005 15084 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:43.161924 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:43.164652 16353 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:43.220088 15229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:43.220763 15229 raft_consensus.cc:606] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:43.223589 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:43.224136 15084 raft_consensus.cc:606] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:43.226617 15158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:43.227083 15158 raft_consensus.cc:606] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:43.229563 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "661df855a98f493ca654519f6be2164f"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:43.230036 15084 raft_consensus.cc:606] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:43.231083 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:02:43.232211 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:43.529390 16573 raft_consensus.cc:993] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:43.530160 16353 raft_consensus.cc:1081] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:43.531896 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:43.532929 15163 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:43.533381 15163 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.568775 15163 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.576866 15163 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:43.580049 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:43.579943 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:43.580981 15080 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.580973 15235 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:43.582825 15235 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:43.583959 15235 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.587025 15235 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.588729 15080 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:43.590161 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:43.591288 16458 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:43.591820 16458 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:43.592741 16458 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.594465 15235 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:43.604025 14985 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 4, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:43.636786 16579 raft_consensus.cc:993] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:43.637387 16579 raft_consensus.cc:1081] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:43.639299 15847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "661df855a98f493ca654519f6be2164f"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:43.639923 15847 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:43.640331 15847 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.648298 15847 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.650848 15847 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:43.651386 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:43.652040 15080 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:43.652346 15080 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:43.653506 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:43.653177 15080 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.654076 15163 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.654827 15080 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.660472 15163 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:43.661765 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:43.662102 15080 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:43.662564 16429 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:43.663036 16429 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:43.663899 16429 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.668184 15235 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:43.675179 14985 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:43.676667 15201 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 30853833ff254a12bcc77a177a964bb9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:43.677078 15163 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:43.681799 15080 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:43.683797 16597 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:43.685267 16353 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.687795 16511 raft_consensus.cc:993] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:43.688315 16511 raft_consensus.cc:1081] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:43.690105 15160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:43.690646 15160 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:43.690979 15160 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.696996 16458 raft_consensus.cc:993] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:43.697978 16533 raft_consensus.cc:1081] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:43.699803 15084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:43.699678 15160 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.700385 15084 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:43.700737 15084 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.702678 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:43.703330 15082 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:43.703619 15082 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:43.704422 15082 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.705546 15082 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.717132 15160 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:43.718107 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:43.722105 15229 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.729565 15084 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.729966 15229 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:43.731153 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:43.732625 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:43.732900 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:43.733225 15158 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:43.733747 15158 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:43.733803 15847 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.734688 15158 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.735822 15158 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.735934 15082 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:02:43.736720 15084 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:43.738268 16458 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:43.739554 16458 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:43.740527 16458 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:43.744889 15847 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:43.743518 16353 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.746436 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:43.747871 16510 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:43.748425 16510 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:43.748610 15163 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.749541 16510 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:43.751737 15158 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:43.754182 15080 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.763362 15084 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:43.771708 14981 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:43.775305 14985 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:43.780992 15052 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe0a67b27bdc4c72abddb618266992a8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:43.782724 15163 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:43.784363 16510 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:43.784372 15229 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:43.791301 16510 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:43.816726 14984 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:43.816893 16511 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.830765 15158 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.847313 15234 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:43.855400 14981 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:43.875767 15080 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:43.877202 15201 consensus_peers.cc:597] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 639c51e1c4da436b99c010fb41cebc63. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:43.877596 16353 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.881763 15158 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:43.884459 16353 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.897166 16597 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.891501 15229 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.900568 15080 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.933692 15160 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:43.934989 15158 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.942839 14981 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:43.966683 15234 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:43.969521 15084 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:43.973587 16533 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:43.983569 16346 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:43.987190 15127 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07a1de4f2ab24624aa87e5619961710d. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:44.011505 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85592.4us
I20260709 13:02:44.131330 15082 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:44.133123 16596 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:44.134708 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 113847us
W20260709 13:02:44.183357 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50391.3us
W20260709 13:02:44.226042 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 91238.9us
I20260709 13:02:44.248554 15235 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:44.254886 15160 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:44.259057 16623 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:44.264408 16629 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:44.268281 16346 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:44.290050 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:44.325459 15084 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:44.328696 15084 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:44.330544 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89757.2us
I20260709 13:02:44.332000 15084 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:44.333701 16533 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:44.334743 16630 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:44.422853 15234 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:02:44.436399 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 143278us
I20260709 13:02:44.466920 15163 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:44.475311 15847 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:44.497557 15235 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:44.500548 16511 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:44.522030 16646 ts_tablet_manager.cc:933] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:44.550432 14981 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:44.573269 16604 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:02:44.573761 16650 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:44.591346 16648 ts_tablet_manager.cc:933] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:44.596807 16647 ts_tablet_manager.cc:933] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:44.619509 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 96172.9us
I20260709 13:02:44.624320 16647 tablet_copy_client.cc:323] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
W20260709 13:02:44.639181 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 102457us
I20260709 13:02:44.725029 15247 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 639c51e1c4da436b99c010fb41cebc63 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:44.725701 15247 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 639c51e1c4da436b99c010fb41cebc63 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-639c51e1c4da436b99c010fb41cebc63
I20260709 13:02:44.726316 16660 ts_tablet_manager.cc:933] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:44.752568 16648 tablet_copy_client.cc:323] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:44.712798 16646 tablet_copy_client.cc:323] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
W20260709 13:02:44.808512 16641 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:02:44.841840 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124607us
W20260709 13:02:44.846743 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89269.8us
I20260709 13:02:44.864289 15247 tablet_copy_source_session.cc:215] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:44.865337 16660 tablet_copy_client.cc:323] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:44.915287 15246 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 30853833ff254a12bcc77a177a964bb9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:44.915983 15246 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 30853833ff254a12bcc77a177a964bb9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-30853833ff254a12bcc77a177a964bb9
W20260709 13:02:44.939605 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55804.3us
I20260709 13:02:44.953150 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 07a1de4f2ab24624aa87e5619961710d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:44.953842 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 07a1de4f2ab24624aa87e5619961710d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-07a1de4f2ab24624aa87e5619961710d
I20260709 13:02:44.997587 16647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 639c51e1c4da436b99c010fb41cebc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:44.999540 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:45.022715 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet fe0a67b27bdc4c72abddb618266992a8 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:45.023569 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet fe0a67b27bdc4c72abddb618266992a8 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-fe0a67b27bdc4c72abddb618266992a8
W20260709 13:02:45.083091 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 187594us
I20260709 13:02:45.135561 15097 tablet_copy_source_session.cc:215] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:45.181133 16647 tablet_copy_client.cc:806] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:45.181847 16647 tablet_copy_client.cc:670] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:45.184473 15173 tablet_copy_source_session.cc:215] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:45.187330 16646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0a67b27bdc4c72abddb618266992a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.192406 16647 tablet_copy_client.cc:538] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:45.212719 16647 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
W20260709 13:02:45.219017 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 87786us
I20260709 13:02:45.275501 15246 tablet_copy_source_session.cc:215] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:45.281481 16648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30853833ff254a12bcc77a177a964bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.286101 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a1de4f2ab24624aa87e5619961710d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.325779 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:45.326531 15163 raft_consensus.cc:606] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:45.332994 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:45.333559 15163 raft_consensus.cc:606] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:45.339471 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f52fa48697b145c98311ef7704e35002"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:45.340058 15080 raft_consensus.cc:606] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:45.342732 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:45.347882 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:45.377784 16672 raft_consensus.cc:993] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:45.378345 16629 raft_consensus.cc:1081] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:45.381246 15082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:45.381783 15082 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:45.382122 15082 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.395874 15082 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.406780 15082 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:45.408123 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:45.409746 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:45.409767 15163 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:45.410221 15847 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.410234 15163 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:45.425817 15847 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:45.428381 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:45.429194 16604 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:45.429661 16604 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:45.430554 16604 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.442471 16647 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:45.448571 16646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59138 (request call id 402) took 1060 ms. Trace:
I20260709 13:02:45.449383 16647 tablet_bootstrap.cc:492] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:45.450260 16647 ts_tablet_manager.cc:1403] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.238s	user 0.021s	sys 0.016s
I20260709 13:02:45.452203 15163 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.465248 15163 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.465389 16647 raft_consensus.cc:359] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.449170 16646 rpcz_store.cc:276] 0709 13:02:44.393856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.418251 (+ 24395us) service_pool.cc:224] Handling call
0709 13:02:44.553215 (+134964us) ts_tablet_manager.cc:934] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
0709 13:02:44.995907 (+442692us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b4c003a15c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3dcb0a20 00007f0d3dcc1a29 00007f0d3dcc213f 00007f0d3dc862ab 00007f0d3dcb3f3c 00007f0d3dcb4461 00007f0d3dcb4768 00007f0d43a3a2eb 00007f0d4495fd3b 00007f0d4495a008 00007f0d44959405
0709 13:02:45.185649 (+189742us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:45.377001 (+191352us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:45.377121 (+   120us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:45.445125 (+ 68004us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:45.445412 (+   287us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:45.445695 (+   283us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:45.448509 (+  2814us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":43745,"spinlock_wait_cycles":10319616,"tablet-copy.queue_time_us":73480,"thread_start_us":186323,"threads_started":2}
I20260709 13:02:45.475164 16646 tablet_copy_client.cc:806] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:45.475735 16646 tablet_copy_client.cc:670] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:45.477819 16647 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:45.479944 16647 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.487092 15847 raft_consensus.cc:1217] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:45.512256 16648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59122 (request call id 309) took 1110 ms. Trace:
I20260709 13:02:45.489969 16647 ts_tablet_manager.cc:1434] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:02:45.497282 16646 tablet_copy_client.cc:538] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:45.523310 15246 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-639c51e1c4da436b99c010fb41cebc63 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:45.523782 15246 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-639c51e1c4da436b99c010fb41cebc63 on tablet 639c51e1c4da436b99c010fb41cebc63 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:45.499634 15163 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:45.521154 16648 rpcz_store.cc:276] 0709 13:02:44.401379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.412716 (+ 11337us) service_pool.cc:224] Handling call
0709 13:02:44.591746 (+179030us) ts_tablet_manager.cc:934] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
0709 13:02:45.280271 (+688525us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:45.354751 (+ 74480us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:45.354873 (+   122us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:45.511429 (+156556us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:45.511699 (+   270us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:45.512080 (+   381us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:45.512152 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":85274,"tablet-copy.queue_time_us":101260,"thread_start_us":111721,"threads_started":2}
I20260709 13:02:45.537670 16648 tablet_copy_client.cc:806] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:45.538239 16648 tablet_copy_client.cc:670] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:45.542402 16648 tablet_copy_client.cc:538] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:45.556049 16648 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:45.557727 16660 tablet_copy_client.cc:806] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:45.560061 16660 tablet_copy_client.cc:670] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:45.562539 14984 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.567097 16660 tablet_copy_client.cc:538] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:45.616078 16648 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:45.617151 16648 tablet_bootstrap.cc:492] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:45.617862 16648 ts_tablet_manager.cc:1403] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.009s
I20260709 13:02:45.626222 16648 raft_consensus.cc:359] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.627285 16648 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:45.628127 16648 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.654515 16604 raft_consensus.cc:993] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:45.655195 16654 raft_consensus.cc:1081] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:45.656751 16648 ts_tablet_manager.cc:1434] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:02:45.658535 15246 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-30853833ff254a12bcc77a177a964bb9 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:45.658973 15246 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-30853833ff254a12bcc77a177a964bb9 on tablet 30853833ff254a12bcc77a177a964bb9 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:45.669008 16626 raft_consensus.cc:993] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:45.669521 16626 raft_consensus.cc:1081] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:45.670888 16660 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:45.627192 16646 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:45.679891 15237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f52fa48697b145c98311ef7704e35002"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:02:45.682332 15237 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:45.682727 15237 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.694958 15237 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:45.701275 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:45.701884 15082 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:45.702184 15082 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:45.702934 15082 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:45.704416 15082 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.717358 15237 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:45.718752 15234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43970
I20260709 13:02:45.719429 15234 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:45.719851 15234 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.729130 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:45.729776 15163 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.729153 15234 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.733239 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:45.733793 15847 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.741644 15163 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:45.742638 15847 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:45.744884 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:45.746541 16353 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:45.747092 16353 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:45.747200 15082 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:45.747946 16353 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.750016 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:45.752028 16634 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:45.752583 16634 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:45.753573 16634 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:45.764334 16646 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:45.765679 16646 tablet_bootstrap.cc:492] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:45.766515 16646 ts_tablet_manager.cc:1403] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.140s	user 0.022s	sys 0.013s
I20260709 13:02:45.768484 15847 raft_consensus.cc:1217] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:02:45.774266 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45669.8us
I20260709 13:02:45.782673 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:45.783329 15161 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:45.783634 15161 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:45.784068 15234 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:45.784386 15161 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:45.786957 15161 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:45.795739 15161 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:45.804389 14985 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.812041 16646 raft_consensus.cc:359] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.813094 16646 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:45.813787 16646 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.826030 16646 ts_tablet_manager.cc:1434] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:02:45.827909 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-fe0a67b27bdc4c72abddb618266992a8 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:45.828418 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-fe0a67b27bdc4c72abddb618266992a8 on tablet fe0a67b27bdc4c72abddb618266992a8 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:45.859812 14984 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.867468 15161 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:45.868990 16579 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:45.890650 16660 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:45.892099 16660 tablet_bootstrap.cc:492] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:45.892953 16660 ts_tablet_manager.cc:1403] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.222s	user 0.031s	sys 0.000s
I20260709 13:02:45.897431 16660 raft_consensus.cc:359] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.898569 16660 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:45.899467 16660 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:45.901872 16660 ts_tablet_manager.cc:1434] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:45.905784 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-07a1de4f2ab24624aa87e5619961710d received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:45.906225 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-07a1de4f2ab24624aa87e5619961710d on tablet 07a1de4f2ab24624aa87e5619961710d with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:45.912297 15847 raft_consensus.cc:1217] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:45.968566 15847 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:45.970356 16579 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:45.987434 15847 raft_consensus.cc:1217] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:46.073549 16353 raft_consensus.cc:1064] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:46.075196 16353 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.083675 15847 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.084915 15154 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.085018 16677 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.086561 16353 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.087930 15080 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.089599 16353 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.107224 16677 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.111146 15847 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.132107 14981 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.135093 15080 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.140276 15154 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.178434 15237 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.181151 16677 raft_consensus.cc:1064] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:46.183493 16677 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.186522 15154 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:46.190605 15847 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:46.193526 16625 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.194859 16617 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.206760 15845 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.208021 15080 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.208153 15155 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:46.209446 16677 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.211867 16353 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.212018 16634 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.216379 16625 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.223802 15154 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.225476 15847 raft_consensus.cc:2955] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.247247 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 639c51e1c4da436b99c010fb41cebc63 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:46.255856 14984 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.258603 16677 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.268471 15846 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.271576 15161 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.274600 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 639c51e1c4da436b99c010fb41cebc63 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:46.287207 15084 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:46.289664 16617 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:46.291911 15080 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.296087 14984 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.299185 15155 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:46.301457 16617 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:46.312150 16698 tablet_replica.cc:333] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:46.316124 16698 raft_consensus.cc:2243] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:46.313784 16346 raft_consensus.cc:1064] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:46.316704 15847 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:46.326349 16346 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.338086 16617 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:02:46.362637 16698 raft_consensus.cc:2272] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:46.374294 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:46.369552 15154 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:46.397239 16625 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:02:46.404676 16698 ts_tablet_manager.cc:1916] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:46.461145 15237 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.472122 15234 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:46.516155 15847 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:46.478749 16707 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 13:02:46.578783 16700 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:02:46.582788 16694 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:02:46.621739 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 55329.7us
I20260709 13:02:46.667932 15847 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:46.723337 15084 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:46.733067 15084 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:46.691429 16698 ts_tablet_manager.cc:1929] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:46.695874 16634 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:02:46.798952 16700 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:46.800915 16723 raft_consensus.cc:1064] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:46.787256 16698 log.cc:1199] T 639c51e1c4da436b99c010fb41cebc63 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/639c51e1c4da436b99c010fb41cebc63
W20260709 13:02:46.807055 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57013.8us
I20260709 13:02:46.816509 16634 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.818997 16617 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.822899 15846 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.804165 16723 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:46.845014 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:46.894840 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 639c51e1c4da436b99c010fb41cebc63 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:46.894260 14985 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.900673 15082 raft_consensus.cc:2955] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:46.937289 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 30853833ff254a12bcc77a177a964bb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:46.995182 15229 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:46.995385 15846 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:46.997778 16711 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.999641 16711 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:47.025897 16709 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.042127 15162 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:47.043644 16702 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:47.045568 15235 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.057703 15084 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.077708 16654 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.094286 15232 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.135185 15232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:47.135871 15232 raft_consensus.cc:606] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:47.161875 15235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "301fa97ba426402092eaf85685303b33"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:47.162485 15235 raft_consensus.cc:606] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:47.162127 15846 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.165292 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a20274fd45144db87032cdf948f324d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:47.165848 15084 raft_consensus.cc:606] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:47.167094 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:47.168690 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:47.170149 16353 raft_consensus.cc:993] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:47.170614 16353 raft_consensus.cc:1081] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:47.172951 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "301fa97ba426402092eaf85685303b33"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57510
I20260709 13:02:47.173540 15163 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:47.173873 15163 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.181020 14983 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.183341 15163 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.188711 15163 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:47.192179 16635 raft_consensus.cc:993] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:47.192690 16635 raft_consensus.cc:1081] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:47.193780 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:47.197294 15084 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.197458 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a20274fd45144db87032cdf948f324d"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:47.198064 15163 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:47.198477 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:47.199033 15235 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:47.198395 15163 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.199311 15235 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:47.202126 15235 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.202138 15161 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.203790 15235 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.206409 15844 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.218736 15235 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:47.220356 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:47.221164 16700 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:47.221671 16700 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:47.222482 16700 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.226415 15084 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:47.232087 15163 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.233364 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 30853833ff254a12bcc77a177a964bb9 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:47.234759 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:47.235334 15084 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:47.235708 15084 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:47.236557 15084 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.243867 15084 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.249754 15163 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:47.242583 15162 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:47.251857 16740 tablet_replica.cc:333] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:47.253305 15084 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:47.254575 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:47.254613 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:47.253387 16740 raft_consensus.cc:2243] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:47.256309 16709 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:47.256337 15234 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.256870 16709 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:47.257771 16709 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:47.258836 15844 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.260444 16669 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:47.256883 16740 raft_consensus.cc:2272] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:47.264943 15080 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.268965 16719 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:47.273532 16740 ts_tablet_manager.cc:1916] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:47.276871 16699 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.278311 14981 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.283843 15234 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:02:47.284017 15080 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.293234 16353 raft_consensus.cc:993] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:47.293854 16727 raft_consensus.cc:1081] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:47.293512 15844 raft_consensus.cc:2955] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.297406 15082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:47.297991 15082 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:47.298429 15082 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.300102 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 07a1de4f2ab24624aa87e5619961710d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:47.305938 16740 ts_tablet_manager.cc:1929] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:47.306767 16740 log.cc:1199] T 30853833ff254a12bcc77a177a964bb9 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/30853833ff254a12bcc77a177a964bb9
I20260709 13:02:47.309175 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 30853833ff254a12bcc77a177a964bb9 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:47.310410 14984 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.312543 15082 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:47.314978 15082 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:47.315559 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:47.316143 15844 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.323388 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:47.324011 15235 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:47.324349 15235 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:47.324453 15844 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:47.326236 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:47.326993 16654 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:47.326905 15235 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:47.327558 16654 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:47.328408 15235 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.328481 16654 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:47.337306 15235 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:47.337553 14981 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.344715 15082 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:47.350867 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a1de4f2ab24624aa87e5619961710d with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:47.354205 16743 tablet_replica.cc:333] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:47.355098 15161 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.355540 15844 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.359527 16743 raft_consensus.cc:2243] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:47.360333 16743 raft_consensus.cc:2272] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:47.361162 16654 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:47.362903 16635 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:47.365563 16743 ts_tablet_manager.cc:1916] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:47.364588 14982 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.370838 16702 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.371202 15161 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.382850 16743 ts_tablet_manager.cc:1929] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:47.383404 16743 log.cc:1199] T 07a1de4f2ab24624aa87e5619961710d P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/07a1de4f2ab24624aa87e5619961710d
I20260709 13:02:47.385131 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 07a1de4f2ab24624aa87e5619961710d (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:47.382246 15845 raft_consensus.cc:2955] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.389017 14985 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.401296 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fe0a67b27bdc4c72abddb618266992a8 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:47.404788 14982 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:47.427237 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet fe0a67b27bdc4c72abddb618266992a8 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:47.428236 16743 tablet_replica.cc:333] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:47.429320 16743 raft_consensus.cc:2243] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:47.429973 16743 raft_consensus.cc:2272] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:47.433774 16743 ts_tablet_manager.cc:1916] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:47.446822 16743 ts_tablet_manager.cc:1929] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:47.447156 16743 log.cc:1199] T fe0a67b27bdc4c72abddb618266992a8 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/fe0a67b27bdc4c72abddb618266992a8
I20260709 13:02:47.448485 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet fe0a67b27bdc4c72abddb618266992a8 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:47.663741 15082 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:47.665068 16669 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:47.670888 15083 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:47.673401 16709 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:47.685854 15235 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:47.696018 16709 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:47.713459 15235 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:47.715391 16699 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:47.735600 15845 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.737036 16654 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:47.741825 15235 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:47.745373 16711 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:48.169647 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.484203 15083 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:48.489650 15234 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:02:48.490681 15052 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca6bbc5df3084af4a938113f16eeedb5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:48.491127 16756 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:48.492641 15163 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:48.493947 16756 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:48.499589 16702 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.505450 15235 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.510107 15163 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.523927 14983 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.532518 15163 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:48.541261 15235 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:48.542822 16721 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:48.543991 15127 consensus_peers.cc:597] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cb4f228ea0b4cc6831089c6ddf8ec96. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:48.545449 15082 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:48.547039 16721 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:48.568013 16669 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.571422 15083 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.573473 15235 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.584031 15234 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:48.589833 14985 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:48.595297 15201 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 30c8fd833bca41f69ed253599a8889b9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:48.597321 15084 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:48.597792 15163 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:48.598994 16713 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:48.601557 16727 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:48.624635 16727 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.626714 15163 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.628674 15084 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.642445 15163 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:48.642964 14983 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.667403 15234 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:48.667403 15084 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:48.668700 16721 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:02:48.669713 15127 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16dab822009a4f00a0a480a48136ae25. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:48.671193 16669 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:48.684770 16669 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.687202 15084 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.693823 15234 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:48.705529 14983 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.737003 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.801306 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:48.801940 15084 raft_consensus.cc:606] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:48.804539 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:48.805099 15163 raft_consensus.cc:606] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:48.808223 15163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de505aa1e895447486f53629c5269998"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:48.808758 15163 raft_consensus.cc:606] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:48.810464 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:48.811693 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:48.980768 16772 ts_tablet_manager.cc:933] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:48.982728 16772 tablet_copy_client.cc:323] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:48.984316 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:48.984870 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-9cb4f228ea0b4cc6831089c6ddf8ec96
I20260709 13:02:48.992106 16721 raft_consensus.cc:993] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:48.992664 15173 tablet_copy_source_session.cc:215] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:48.992715 16721 raft_consensus.cc:1081] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:02:48.994714 15234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43970
I20260709 13:02:48.995342 15234 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.995718 15234 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.996213 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:49.003919 15234 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.009825 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:49.009984 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:49.009997 15234 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:49.010530 15084 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.010710 15163 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:49.011148 15163 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:49.011931 15163 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.034502 15163 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.039608 15084 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:49.041426 15202 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:49.042696 16713 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:49.043655 16713 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:49.044646 16713 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.060599 15163 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:49.060240 16772 tablet_copy_client.cc:806] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:49.061743 16772 tablet_copy_client.cc:670] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:49.066387 14982 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:49.069669 16772 tablet_copy_client.cc:538] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:49.082638 16772 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:49.104054 16772 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:49.105147 16772 tablet_bootstrap.cc:492] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:49.105906 16772 ts_tablet_manager.cc:1403] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.008s
I20260709 13:02:49.108942 16772 raft_consensus.cc:359] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.109863 16772 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:49.110522 16772 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.113081 16772 ts_tablet_manager.cc:1434] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:49.115183 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-9cb4f228ea0b4cc6831089c6ddf8ec96 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:49.115597 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-9cb4f228ea0b4cc6831089c6ddf8ec96 on tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:49.116783 16776 ts_tablet_manager.cc:933] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:49.133285 16776 tablet_copy_client.cc:323] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:49.134850 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet ca6bbc5df3084af4a938113f16eeedb5 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:49.135367 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet ca6bbc5df3084af4a938113f16eeedb5 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-ca6bbc5df3084af4a938113f16eeedb5
I20260709 13:02:49.142812 15097 tablet_copy_source_session.cc:215] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:49.146540 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6bbc5df3084af4a938113f16eeedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:49.150286 16772 ts_tablet_manager.cc:933] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:49.153878 16669 raft_consensus.cc:993] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:49.154590 16669 raft_consensus.cc:1081] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:49.156728 15084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de505aa1e895447486f53629c5269998"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:49.157352 15084 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:49.157791 15084 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.186542 16772 tablet_copy_client.cc:323] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.195:42229
I20260709 13:02:49.188285 16776 tablet_copy_client.cc:806] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:49.189061 15246 tablet_copy_service.cc:140] P 4b633b84351f40de9437e3d532fe6fa7: Received BeginTabletCopySession request for tablet 30c8fd833bca41f69ed253599a8889b9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:41710)
I20260709 13:02:49.189675 15246 tablet_copy_service.cc:161] P 4b633b84351f40de9437e3d532fe6fa7: Beginning new tablet copy session on tablet 30c8fd833bca41f69ed253599a8889b9 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:41710: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-30c8fd833bca41f69ed253599a8889b9
I20260709 13:02:49.189148 16776 tablet_copy_client.cc:670] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:49.205135 16779 ts_tablet_manager.cc:933] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:49.208230 15246 tablet_copy_source_session.cc:215] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:49.212633 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30c8fd833bca41f69ed253599a8889b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:49.214047 15084 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:49.215725 16776 tablet_copy_client.cc:538] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:49.229614 15084 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:49.229775 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:49.230677 15163 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:49.230960 15163 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:49.231666 15163 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:49.233965 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:49.234531 15235 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.232674 15163 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.241257 16654 raft_consensus.cc:993] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:49.242445 16702 raft_consensus.cc:1081] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:49.242879 15163 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:49.244068 15844 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:49.244398 15054 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:49.244665 15844 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:49.245272 15844 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.247267 16654 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:49.247794 16654 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:49.248570 16654 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:49.249306 15235 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:49.250743 16776 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:49.259042 15844 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:49.265268 16779 tablet_copy_client.cc:323] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:49.267949 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:49.270272 15084 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:49.270663 15084 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:49.271770 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:49.271586 15084 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:49.272359 15163 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.271955 14982 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:49.273368 15084 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.291174 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 16dab822009a4f00a0a480a48136ae25 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:49.291718 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 16dab822009a4f00a0a480a48136ae25 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-16dab822009a4f00a0a480a48136ae25
I20260709 13:02:49.294500 15163 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:49.295876 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:49.312775 16693 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:49.299157 15173 tablet_copy_source_session.cc:215] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:49.313392 16693 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:49.314452 16693 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:49.296214 15844 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:49.317028 16779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16dab822009a4f00a0a480a48136ae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:49.295385 16772 tablet_copy_client.cc:806] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:49.329466 16772 tablet_copy_client.cc:670] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:49.339596 16776 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:49.340925 16776 tablet_bootstrap.cc:492] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:49.341764 16776 ts_tablet_manager.cc:1403] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.016s
I20260709 13:02:49.329972 15084 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:49.346061 16772 tablet_copy_client.cc:538] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:49.350297 16776 raft_consensus.cc:359] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.352061 16776 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:49.352993 16776 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.356652 16776 ts_tablet_manager.cc:1434] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:02:49.358486 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ca6bbc5df3084af4a938113f16eeedb5 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:49.358945 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ca6bbc5df3084af4a938113f16eeedb5 on tablet ca6bbc5df3084af4a938113f16eeedb5 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:49.381198 16772 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:49.384663 14982 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 2, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:49.395704 16779 tablet_copy_client.cc:806] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:49.396742 16779 tablet_copy_client.cc:670] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:49.405433 16779 tablet_copy_client.cc:538] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:49.410272 16772 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:49.420600 16779 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:49.411353 16772 tablet_bootstrap.cc:492] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:49.423586 16772 ts_tablet_manager.cc:1403] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.043s	user 0.027s	sys 0.000s
I20260709 13:02:49.426677 16772 raft_consensus.cc:359] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.427601 16772 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:49.429525 16772 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.432224 16772 ts_tablet_manager.cc:1434] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:02:49.435945 15246 tablet_copy_service.cc:342] P 4b633b84351f40de9437e3d532fe6fa7: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-30c8fd833bca41f69ed253599a8889b9 received from {username='slave'} at 127.0.0.1:41710
I20260709 13:02:49.436354 15246 tablet_copy_service.cc:434] P 4b633b84351f40de9437e3d532fe6fa7: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-30c8fd833bca41f69ed253599a8889b9 on tablet 30c8fd833bca41f69ed253599a8889b9 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:49.461470 16779 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:49.462662 16779 tablet_bootstrap.cc:492] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:49.466177 16779 ts_tablet_manager.cc:1403] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.046s	user 0.026s	sys 0.004s
I20260709 13:02:49.469605 16779 raft_consensus.cc:359] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.470356 16779 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:49.470932 16779 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:49.473680 16779 ts_tablet_manager.cc:1434] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:02:49.476538 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-16dab822009a4f00a0a480a48136ae25 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:49.476976 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-16dab822009a4f00a0a480a48136ae25 on tablet 16dab822009a4f00a0a480a48136ae25 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:49.497648 15844 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Advancing to term 2
I20260709 13:02:49.503554 15844 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:49.505136 16713 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:49.514011 16713 raft_consensus.cc:1064] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
W20260709 13:02:49.514488 16713 raft_consensus.cc:1068] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Unable to promote non-voter 3c28be6f86ea4fb6b79a47d3999f5a82: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:49.597404 15084 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:49.599906 16713 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:49.600603 15844 raft_consensus.cc:1217] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:49.639271 15163 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:49.648063 16796 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:49.667171 16713 raft_consensus.cc:1064] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:49.669742 16713 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:49.672533 15844 raft_consensus.cc:1217] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:49.687994 15844 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:49.689790 15084 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:49.693186 16795 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:49.695140 16713 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:02:49.702014 15162 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:49.703446 15844 raft_consensus.cc:1217] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:02:49.707031 15162 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:49.707505 16782 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:49.708817 16754 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:49.711534 15084 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:49.718822 16693 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:49.731503 16774 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:49.779227 15844 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:49.795032 15232 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:49.798134 16654 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:49.798949 15161 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:49.802399 15084 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:49.809363 16796 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:49.812510 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:49.827873 14985 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 2 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:49.858414 15163 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.160356 16721 raft_consensus.cc:1064] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:50.163249 16721 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.172976 15845 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.173861 15232 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.174023 15084 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.174715 16721 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.175776 16699 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.176527 16669 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.201797 16699 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.203512 15845 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.208407 15084 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.210775 16713 raft_consensus.cc:1064] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:50.213061 16713 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.227916 14982 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.231992 15845 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:50.232872 15163 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:50.233500 16797 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.234351 16795 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.238236 15084 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:50.239881 16795 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.241528 15235 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.265286 15163 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.269225 16796 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.270929 15846 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.283782 15845 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.285504 15083 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.286681 16669 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.291406 16654 raft_consensus.cc:1064] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:50.299361 16654 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.291165 16721 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:02:50.344591 16669 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.350184 15161 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.374186 15083 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.416823 15845 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.423063 15229 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.421088 15084 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.412431 14982 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.435582 16654 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.451471 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 16dab822009a4f00a0a480a48136ae25 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:50.459777 14982 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:50.455968 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 145805us
I20260709 13:02:50.463994 16817 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:50.477730 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 16dab822009a4f00a0a480a48136ae25 with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:50.482650 16827 tablet_replica.cc:333] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:50.491667 15155 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:50.499181 15845 raft_consensus.cc:2955] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.510075 16654 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:50.521777 16827 raft_consensus.cc:2243] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.522743 16827 raft_consensus.cc:2272] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.521581 16654 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.523113 15846 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.526515 16827 ts_tablet_manager.cc:1916] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:50.532011 15234 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.541667 15844 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:50.544590 16825 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.544818 15083 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:50.549371 16795 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:50.553447 15155 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.557312 15234 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.568938 16796 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.576999 14985 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.579720 15084 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.589435 16827 ts_tablet_manager.cc:1929] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:50.595851 16827 log.cc:1199] T 16dab822009a4f00a0a480a48136ae25 P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/16dab822009a4f00a0a480a48136ae25
I20260709 13:02:50.597909 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 16dab822009a4f00a0a480a48136ae25 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:50.603066 15846 raft_consensus.cc:2955] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.649041 15083 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:50.654495 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 44752us
I20260709 13:02:50.664687 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 30c8fd833bca41f69ed253599a8889b9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:50.669553 15229 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:50.677335 15161 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.678056 15846 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.679852 15845 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.673707 14984 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.681011 16796 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:50.690656 16825 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.703532 16825 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.685652 15234 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:50.687044 16816 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.712739 16816 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.714794 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet 30c8fd833bca41f69ed253599a8889b9 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:50.717125 16830 tablet_replica.cc:333] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:50.718135 16830 raft_consensus.cc:2243] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.718927 16830 raft_consensus.cc:2272] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.719494 16818 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.721356 14970 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:50.719521 15845 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.735625 15846 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.736490 15234 raft_consensus.cc:2955] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.748905 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca6bbc5df3084af4a938113f16eeedb5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:50.747674 15161 raft_consensus.cc:2955] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:50.755668 14984 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 4 to 5, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.758319 14983 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 4 to 5, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.763402 16830 ts_tablet_manager.cc:1916] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:50.782157 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet ca6bbc5df3084af4a938113f16eeedb5 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:50.783067 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:50.798012 16831 tablet_replica.cc:333] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:50.799074 16831 raft_consensus.cc:2243] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.799960 16831 raft_consensus.cc:2272] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.804522 16831 ts_tablet_manager.cc:1916] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:50.812528 16830 ts_tablet_manager.cc:1929] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:02:50.813138 16830 log.cc:1199] T 30c8fd833bca41f69ed253599a8889b9 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/30c8fd833bca41f69ed253599a8889b9
I20260709 13:02:50.814808 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet 30c8fd833bca41f69ed253599a8889b9 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:50.814982 16830 tablet_replica.cc:333] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:50.816148 16830 raft_consensus.cc:2243] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.816958 16830 raft_consensus.cc:2272] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.821082 16830 ts_tablet_manager.cc:1916] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:50.821851 16831 ts_tablet_manager.cc:1929] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:50.822338 16831 log.cc:1199] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/9cb4f228ea0b4cc6831089c6ddf8ec96
I20260709 13:02:50.823954 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 9cb4f228ea0b4cc6831089c6ddf8ec96 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:50.833468 16830 ts_tablet_manager.cc:1929] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:50.833894 16830 log.cc:1199] T ca6bbc5df3084af4a938113f16eeedb5 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/ca6bbc5df3084af4a938113f16eeedb5
I20260709 13:02:50.835165 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet ca6bbc5df3084af4a938113f16eeedb5 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:50.933272 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.977272 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:50.977834 15084 raft_consensus.cc:606] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:50.980294 15234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4010b839395412ca61c77e701fcb965"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:02:50.980791 15234 raft_consensus.cc:606] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:50.981851 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:50.983024 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:51.513195 16796 raft_consensus.cc:993] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:51.513748 16825 raft_consensus.cc:1081] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:51.515111 15084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4010b839395412ca61c77e701fcb965"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40386
I20260709 13:02:51.515633 15084 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:51.515866 15084 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:51.520282 15084 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:51.521796 15084 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:51.522511 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:51.522671 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:51.523134 15155 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:51.523303 15234 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:51.523679 15234 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:02:51.524333 15234 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:51.525398 15234 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:51.529400 15155 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:51.530712 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:51.531517 16834 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:51.531991 16834 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:51.533282 15234 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:51.532910 16834 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:51.545234 14984 catalog_manager.cc:5697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.573591 16818 raft_consensus.cc:993] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:51.574154 16818 raft_consensus.cc:1081] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Signalling peer 3c28be6f86ea4fb6b79a47d3999f5a82 to start an election
I20260709 13:02:51.575852 15845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:59138
I20260709 13:02:51.576349 15845 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:51.576661 15845 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:51.582245 15845 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:51.584213 15845 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:51.585019 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:51.585270 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:51.585640 15084 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:51.585844 15161 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:51.585999 15084 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:51.586745 15084 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:51.587625 15084 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:51.590660 15161 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:51.592223 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:51.592747 15084 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:51.593112 16832 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:51.593641 16832 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:51.594478 16832 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:51.604044 14984 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:51.859880 15161 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:02:51.866305 15127 consensus_peers.cc:597] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4150ed0096c74899b55a5128304e91f6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:51.866588 15083 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:51.866616 15234 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:51.867925 16841 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:51.868685 16823 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:51.876314 16841 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.878113 15234 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.886576 15083 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.894562 14984 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.895416 15084 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:51.907955 15161 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:51.911929 15052 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af12fc69de0b47b9adaf0267fb143776. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:51.913148 16834 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:51.917605 15229 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:51.919663 16834 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:51.931758 16834 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.936379 15229 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.953019 14985 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.954681 15163 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:51.958179 15234 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:51.959571 16834 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:51.948185 15161 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:51.972143 15234 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:51.972311 15084 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:51.973951 16823 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:51.975646 16699 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:02:51.978224 15127 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7defae86b18344bdb8b04e0d7c8f1a8d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:51.985536 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:52.009195 16823 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.016144 15163 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:52.018371 16847 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:52.012022 15234 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.035154 15084 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.058049 15083 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.066957 14985 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.072260 15234 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:52.073160 15155 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:02:52.073515 15052 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de505aa1e895447486f53629c5269998. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:52.080268 16847 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:52.086337 16834 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:52.096266 16847 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.104629 15234 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.114622 15155 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.131042 14984 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index -1 to 4, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:52.159566 15083 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:52.161649 16832 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:52.179759 15155 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:52.188889 16832 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:52.371706 16866 ts_tablet_manager.cc:933] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:52.387531 16866 tablet_copy_client.cc:323] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:52.389799 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 4150ed0096c74899b55a5128304e91f6 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:52.390431 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 4150ed0096c74899b55a5128304e91f6 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-4150ed0096c74899b55a5128304e91f6
I20260709 13:02:52.400198 15173 tablet_copy_source_session.cc:215] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:52.406180 16866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150ed0096c74899b55a5128304e91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.427954 16866 tablet_copy_client.cc:806] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:52.428723 16866 tablet_copy_client.cc:670] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:52.429032 16868 ts_tablet_manager.cc:933] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:52.438639 16866 tablet_copy_client.cc:538] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:52.449657 16866 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:52.463651 16868 tablet_copy_client.cc:323] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:52.514431 16866 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:52.515543 16866 tablet_bootstrap.cc:492] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:52.516412 16866 ts_tablet_manager.cc:1403] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.022s
I20260709 13:02:52.520927 16866 raft_consensus.cc:359] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.521852 16866 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:52.522710 16866 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.526307 16866 ts_tablet_manager.cc:1434] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:02:52.624668 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 7defae86b18344bdb8b04e0d7c8f1a8d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:52.625236 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 7defae86b18344bdb8b04e0d7c8f1a8d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-7defae86b18344bdb8b04e0d7c8f1a8d
I20260709 13:02:52.655968 15173 tablet_copy_source_session.cc:215] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:52.685640 16868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7defae86b18344bdb8b04e0d7c8f1a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.699672 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-4150ed0096c74899b55a5128304e91f6 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:52.700161 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-4150ed0096c74899b55a5128304e91f6 on tablet 4150ed0096c74899b55a5128304e91f6 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:52.731179 16866 ts_tablet_manager.cc:933] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:52.748122 16866 tablet_copy_client.cc:323] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:52.749917 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet af12fc69de0b47b9adaf0267fb143776 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:52.750501 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet af12fc69de0b47b9adaf0267fb143776 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-af12fc69de0b47b9adaf0267fb143776
I20260709 13:02:52.760553 16876 ts_tablet_manager.cc:933] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:52.768144 15097 tablet_copy_source_session.cc:215] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:52.773200 16868 tablet_copy_client.cc:806] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:52.830222 16876 tablet_copy_client.cc:323] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:02:52.833768 15098 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet de505aa1e895447486f53629c5269998 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:02:52.834339 15098 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet de505aa1e895447486f53629c5269998 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-de505aa1e895447486f53629c5269998
I20260709 13:02:52.844118 15098 tablet_copy_source_session.cc:215] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:52.847011 16868 tablet_copy_client.cc:670] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:52.855732 16876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de505aa1e895447486f53629c5269998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.861188 16868 tablet_copy_client.cc:538] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:52.877681 15845 raft_consensus.cc:1217] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 13:02:52.884528 16876 tablet_copy_client.cc:806] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:52.885291 16876 tablet_copy_client.cc:670] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:52.891755 16866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af12fc69de0b47b9adaf0267fb143776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.891932 16868 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:52.906876 16876 tablet_copy_client.cc:538] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:52.921768 16866 tablet_copy_client.cc:806] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:52.922497 16866 tablet_copy_client.cc:670] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:52.922801 16876 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:52.926334 16866 tablet_copy_client.cc:538] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:52.941942 16866 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:52.957521 16876 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:52.958549 16876 tablet_bootstrap.cc:492] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:52.959324 16876 ts_tablet_manager.cc:1403] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.012s
I20260709 13:02:52.962443 16868 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:52.962636 16876 raft_consensus.cc:359] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.963495 16868 tablet_bootstrap.cc:492] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:52.963616 16876 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:52.964321 16868 ts_tablet_manager.cc:1403] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.010s
I20260709 13:02:52.964686 16876 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.967769 16876 ts_tablet_manager.cc:1434] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:52.967489 16868 raft_consensus.cc:359] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.968988 16868 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:52.969656 16868 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.970625 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-de505aa1e895447486f53629c5269998 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:52.971081 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-de505aa1e895447486f53629c5269998 on tablet de505aa1e895447486f53629c5269998 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:52.973457 16868 ts_tablet_manager.cc:1434] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:02:52.980207 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-7defae86b18344bdb8b04e0d7c8f1a8d received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:52.980687 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-7defae86b18344bdb8b04e0d7c8f1a8d on tablet 7defae86b18344bdb8b04e0d7c8f1a8d with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:52.983963 16866 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:52.984993 16866 tablet_bootstrap.cc:492] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:52.985714 16866 ts_tablet_manager.cc:1403] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.010s
I20260709 13:02:52.988621 16866 raft_consensus.cc:359] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.989342 16866 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:52.989929 16866 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:52.992002 16866 ts_tablet_manager.cc:1434] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:02:52.994594 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-af12fc69de0b47b9adaf0267fb143776 received from {username='slave'} at 127.0.0.1:56652
I20260709 13:02:52.995056 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-af12fc69de0b47b9adaf0267fb143776 on tablet af12fc69de0b47b9adaf0267fb143776 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:53.006325 15845 raft_consensus.cc:1217] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:53.097905 15845 raft_consensus.cc:1217] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:53.110656 15845 raft_consensus.cc:1217] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:02:53.179998 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:53.240645 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:53.241283 15084 raft_consensus.cc:606] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:53.242641 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:02:53.244230 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:53.274957 16860 raft_consensus.cc:993] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:53.275596 16871 raft_consensus.cc:1081] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:02:53.277168 15161 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:02:53.277727 15161 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.278030 15161 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:53.284399 15161 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.287066 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:53.287686 15084 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:53.287932 15161 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:53.288025 15084 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:53.288946 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:53.289573 15845 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:53.288955 15084 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.291190 15084 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:53.298893 15845 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:02:53.300230 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:02:53.302138 15084 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:02:53.303231 16853 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:53.304422 16853 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:53.305199 16853 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.319422 14982 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 3, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 3 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.362998 16853 raft_consensus.cc:1064] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:53.365236 16853 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.371742 15844 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:53.372337 15234 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:53.373769 16841 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:53.375089 16823 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:53.377521 15084 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:53.379876 16889 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:53.414928 16841 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.421479 15083 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.434238 15229 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.434008 15844 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:53.504765 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080028bac0 after lost signal to thread 13970
W20260709 13:02:53.506863 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080028c200 after lost signal to thread 15020
W20260709 13:02:53.510995 15109 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0080 after lost signal to thread 15112
W20260709 13:02:53.513253 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080028b9c0 after lost signal to thread 15187
W20260709 13:02:53.515512 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800338fa0 after lost signal to thread 15261
W20260709 13:02:53.518741 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080035ac00 after lost signal to thread 15872
I20260709 13:02:53.519164 14982 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 5 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } attrs { replace: true } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.552181 16889 raft_consensus.cc:1064] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:53.559208 16889 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.587864 15846 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:53.590119 15083 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:53.591459 16834 raft_consensus.cc:1064] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:53.591840 15234 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:53.592211 16841 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:53.593654 16853 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:53.649603 16823 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:53.667783 16896 raft_consensus.cc:1064] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:02:53.630236 16834 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.736124 16896 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.761445 16841 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.873618 15155 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:53.882550 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 88924.7us
I20260709 13:02:53.887674 15084 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.896824 15232 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.901413 15083 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:53.900480 14982 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:53.911784 16906 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:53.929215 15161 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:53.934119 15846 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:53.936048 16860 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:53.947163 15082 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:53.952721 15845 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:53.955440 16841 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:53.957803 15845 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:53.968394 15846 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:53.968873 15161 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:53.969084 15842 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:53.974548 16823 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:53.974704 16895 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:53.980275 16902 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:53.985388 15235 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:53.993271 16841 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.002410 15087 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.006371 15162 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:54.007304 15843 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:54.004107 16898 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:54.023568 16898 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:54.020808 16823 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:02:54.026331 15235 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:54.039724 15234 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:54.046681 16847 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:02:54.039927 15841 raft_consensus.cc:2955] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.047960 16853 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:02:54.054817 15235 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.101511 16898 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
W20260709 13:02:54.222707 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74694.7us
I20260709 13:02:54.170768 15847 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.054912 16889 raft_consensus.cc:2955] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.247987 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:54.224030 16891 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:02:54.328935 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 74462.4us
I20260709 13:02:54.352140 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7defae86b18344bdb8b04e0d7c8f1a8d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:54.382506 14985 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 6 to 7, VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) evicted. New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:54.393424 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55429.9us
I20260709 13:02:54.391219 16871 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.405157 14973 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4150ed0096c74899b55a5128304e91f6 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:02:54.412040 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44410.1us
W20260709 13:02:54.474115 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 244743us
W20260709 13:02:54.485144 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58177.2us
I20260709 13:02:54.496294 16930 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:02:54.506897 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77698us
W20260709 13:02:54.509740 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82294.2us
I20260709 13:02:54.529266 15161 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.535746 15162 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.560071 15844 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.576188 15237 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.576093 15842 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.627636 15229 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.657694 14984 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 4 to 5, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.657712 15071 tablet_service.cc:1558] Processing DeleteTablet for tablet 4150ed0096c74899b55a5128304e91f6 with delete_type TABLET_DATA_TOMBSTONED (TS 12daa67457954a0cab27e32bf85fa115 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40384
I20260709 13:02:54.662876 14985 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 4 to 5, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.665227 14985 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 2 to 3, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } attrs { replace: true } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.675928 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet 7defae86b18344bdb8b04e0d7c8f1a8d with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43954
I20260709 13:02:54.695230 16942 tablet_replica.cc:333] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:02:54.702322 16942 raft_consensus.cc:2243] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:54.702719 16943 tablet_replica.cc:333] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:02:54.704104 16942 raft_consensus.cc:2272] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:54.705188 16943 raft_consensus.cc:2243] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:54.707670 16942 ts_tablet_manager.cc:1916] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:54.708668 16943 raft_consensus.cc:2272] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:54.714573 16943 ts_tablet_manager.cc:1916] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:54.730336 15083 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:54.743899 15229 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:54.744556 15844 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:54.746132 16871 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:54.747287 16924 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:54.750968 15087 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:54.757788 16930 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.761749 15842 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.762159 15229 raft_consensus.cc:2955] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.762655 16942 ts_tablet_manager.cc:1929] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 13:02:54.773214 16942 log.cc:1199] T 4150ed0096c74899b55a5128304e91f6 P 12daa67457954a0cab27e32bf85fa115: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/wals/4150ed0096c74899b55a5128304e91f6
I20260709 13:02:54.776721 14972 catalog_manager.cc:5028] TS 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): tablet 4150ed0096c74899b55a5128304e91f6 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:54.786101 15847 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:54.787221 15232 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:54.791851 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet de505aa1e895447486f53629c5269998 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:54.791663 16930 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:54.803005 16834 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:54.806867 16871 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.807426 14984 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: config changed from index 5 to 6, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.809633 15237 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.832170 14972 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet af12fc69de0b47b9adaf0267fb143776 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:54.836081 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet de505aa1e895447486f53629c5269998 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:54.868181 16943 ts_tablet_manager.cc:1929] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:02:54.868829 16943 log.cc:1199] T 7defae86b18344bdb8b04e0d7c8f1a8d P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/7defae86b18344bdb8b04e0d7c8f1a8d
I20260709 13:02:54.871488 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet 7defae86b18344bdb8b04e0d7c8f1a8d (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:54.883136 16945 tablet_replica.cc:333] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:54.882164 14983 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New cstate: current_term: 1 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.902218 15844 raft_consensus.cc:2955] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:02:54.903821 16945 raft_consensus.cc:2243] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:54.904742 16945 raft_consensus.cc:2272] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:54.912009 15151 tablet_service.cc:1558] Processing DeleteTablet for tablet af12fc69de0b47b9adaf0267fb143776 with delete_type TABLET_DATA_TOMBSTONED (TS 3d45101e0851484690b4f03a4dcdb63d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57482
I20260709 13:02:54.923749 16945 ts_tablet_manager.cc:1916] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:55.055663 16945 ts_tablet_manager.cc:1929] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:02:55.056183 16945 log.cc:1199] T de505aa1e895447486f53629c5269998 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/de505aa1e895447486f53629c5269998
I20260709 13:02:55.058314 16945 tablet_replica.cc:333] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:02:55.059397 16945 raft_consensus.cc:2243] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:55.060105 16945 raft_consensus.cc:2272] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:55.062844 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet de505aa1e895447486f53629c5269998 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:55.078511 16945 ts_tablet_manager.cc:1916] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:55.116660 16945 ts_tablet_manager.cc:1929] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:55.117093 16945 log.cc:1199] T af12fc69de0b47b9adaf0267fb143776 P 3d45101e0851484690b4f03a4dcdb63d: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/wals/af12fc69de0b47b9adaf0267fb143776
I20260709 13:02:55.118606 14973 catalog_manager.cc:5028] TS 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): tablet af12fc69de0b47b9adaf0267fb143776 (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:02:55.603428 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:55.650368 15162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:53006
I20260709 13:02:55.651194 15162 raft_consensus.cc:606] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:55.653940 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:02:55.655957 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:55.805732 16925 raft_consensus.cc:993] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:55.806309 16853 raft_consensus.cc:1081] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:02:55.808171 15083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:40400
I20260709 13:02:55.808955 15083 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:55.809284 15083 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:55.814538 15083 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:55.817039 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:55.816798 15083 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:55.817801 15162 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:55.818262 15162 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:02:55.818364 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:02:55.818990 15232 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:55.819833 15162 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:55.821099 15162 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:55.825577 15232 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:55.826778 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:55.827255 15162 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:55.827451 16834 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:55.828061 16834 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:02:55.829290 16834 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:55.837982 14984 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:55.972751 15161 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:55.983644 15232 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:02:55.984092 15127 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 301fa97ba426402092eaf85685303b33. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.985314 16823 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:55.987986 15083 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:02:55.989574 16823 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:56.018064 16925 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:56.023128 15229 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:56.025624 15082 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:02:56.047453 14984 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 3, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.237900 15161 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:56.239012 16834 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:56.257480 15229 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:56.261981 16960 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:56.445699 16967 ts_tablet_manager.cc:933] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:56.451536 16967 tablet_copy_client.cc:323] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.194:38049
I20260709 13:02:56.453524 15173 tablet_copy_service.cc:140] P 3d45101e0851484690b4f03a4dcdb63d: Received BeginTabletCopySession request for tablet 301fa97ba426402092eaf85685303b33 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:54628)
I20260709 13:02:56.454020 15173 tablet_copy_service.cc:161] P 3d45101e0851484690b4f03a4dcdb63d: Beginning new tablet copy session on tablet 301fa97ba426402092eaf85685303b33 from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:54628: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-301fa97ba426402092eaf85685303b33
I20260709 13:02:56.460661 15173 tablet_copy_source_session.cc:215] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:56.464015 16967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301fa97ba426402092eaf85685303b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:56.480582 16967 tablet_copy_client.cc:806] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:56.481236 16967 tablet_copy_client.cc:670] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:56.486207 16967 tablet_copy_client.cc:538] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:56.495623 16967 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:02:56.521831 16967 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:56.522704 16967 tablet_bootstrap.cc:492] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:02:56.523314 16967 ts_tablet_manager.cc:1403] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.009s
I20260709 13:02:56.525811 16967 raft_consensus.cc:359] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:56.526492 16967 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:02:56.527078 16967 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:56.529814 16967 ts_tablet_manager.cc:1434] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:02:56.532111 15173 tablet_copy_service.cc:342] P 3d45101e0851484690b4f03a4dcdb63d: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-301fa97ba426402092eaf85685303b33 received from {username='slave'} at 127.0.0.1:54628
I20260709 13:02:56.532546 15173 tablet_copy_service.cc:434] P 3d45101e0851484690b4f03a4dcdb63d: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-301fa97ba426402092eaf85685303b33 on tablet 301fa97ba426402092eaf85685303b33 with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:56.657003 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:56.903867 15842 raft_consensus.cc:1217] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:02:57.786366 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:58.094702 13972 debug-util.cc:398] Leaking SignalData structure 0x7b080000f780 after lost signal to thread 15127
W20260709 13:02:58.095402 13972 kernel_stack_watchdog.cc:198] Thread 15127 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 137ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:02:58.213276 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.271s	user 0.014s	sys 0.005s
W20260709 13:02:58.213743 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.014s	sys 0.005s
W20260709 13:02:58.308799 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.35229e+06us
I20260709 13:02:58.340780 16981 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.341459 16981 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.344019 16982 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:02:58.344702 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.403s	user 0.011s	sys 0.010s
I20260709 13:02:58.344566 16982 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.345016 16980 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.345835 16980 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:58.345052 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.403s	user 0.012s	sys 0.010s
W20260709 13:02:58.349445 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 1.40755e+06us
I20260709 13:02:58.351649 16987 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.352500 16981 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.353336 16987 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:58.354568 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.006s	sys 0.008s
W20260709 13:02:58.354974 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.205s	user 0.006s	sys 0.008s
I20260709 13:02:58.357659 16978 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.357869 16977 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.358315 16978 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.358436 16977 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.359257 16980 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.360633 16980 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.361023 16977 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.361109 16980 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.363615 16980 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.354658 16982 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.368510 16978 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.369817 16987 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.378798 16982 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.379396 16982 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.380388 16985 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.381433 16993 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.381815 16988 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.381990 16993 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.383075 16991 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.383651 16991 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.382822 16988 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.381462 16982 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.382299 16990 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.385109 16993 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.385365 16990 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.387771 16990 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.380930 16985 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.388983 16990 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.389096 16995 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.389647 16990 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.389734 16995 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.389650 16991 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.390748 16985 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.392314 16990 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.392617 16995 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.396011 16991 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.396242 16987 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.396760 16987 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.396710 16991 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.398849 16987 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.399714 16991 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.400166 16985 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.400362 16987 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.400879 16985 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.401793 16995 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.402380 16995 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.402719 16990 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.403221 16990 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.403676 16985 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.404517 16995 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.400938 16987 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.406090 16990 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.387586 16988 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.406792 16987 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.417266 16994 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.417879 16994 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.420171 16994 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.421074 16987 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.421706 16987 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.422287 16994 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.422843 16994 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.424314 16987 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.424386 16992 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.425091 16992 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.426287 16994 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.427217 16992 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.428771 16992 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.429563 16992 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.431792 16992 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.433231 16992 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.434291 16987 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.434361 16992 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.433506 16982 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.436041 16982 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.439067 16976 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.439851 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.439616 16976 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:58.441220 15814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.000s	sys 0.014s
W20260709 13:02:58.441706 15814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.291s	user 0.001s	sys 0.014s
I20260709 13:02:58.442189 16976 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.442580 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.442729 16976 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.443177 16976 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.444831 16993 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.445415 16993 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.446883 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 301) took 1493 ms. Trace:
I20260709 13:02:58.447188 16978 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.447959 16993 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.447906 16978 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.449234 16996 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.447305 15161 rpcz_store.cc:276] 0709 13:02:56.953971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.446168 (+1492197us) service_pool.cc:224] Handling call
0709 13:02:58.446425 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.446592 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.446620 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:58.446660 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:58.446690 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.446739 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.446834 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.450067 16996 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.450773 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:58.451370 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.003s	sys 0.006s
I20260709 13:02:58.451560 16978 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:58.441495 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.452325 15082 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:58.453696 16993 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.453706 16996 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.454967 16993 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.435075 16987 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.454262 16977 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.456045 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.456712 15082 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:58.456561 16977 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.457749 16987 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:58.451741 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.301s	user 0.004s	sys 0.006s
I20260709 13:02:58.458652 16977 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.459232 16977 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.459709 16977 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.460973 16990 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.461503 16990 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:58.463581 16990 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.464028 16990 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.460037 17002 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.465178 16983 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.465194 17002 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.465747 16983 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.466012 16977 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.468495 16980 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.469460 16989 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.469705 16980 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.470067 16989 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.472234 16980 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.467007 16988 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.467294 16993 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.466647 16979 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.433799 16994 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.474071 16988 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.476383 16946 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.478437 16979 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.481217 15083 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:58.484514 16979 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.485014 16980 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.464444 16990 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.485044 16946 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.487749 16992 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.486747 16982 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.485474 16983 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.490319 16980 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.489507 16946 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.490319 16976 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.474884 16989 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.489481 16990 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.493615 16946 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.494119 16946 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.496289 16946 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:02:58.502434 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52428.8us
I20260709 13:02:58.504391 16946 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.504966 16946 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.505213 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.506079 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.506570 16976 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.506996 16976 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.507040 16946 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.507450 16976 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.508109 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.508719 15087 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:58.507660 16946 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.511379 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.510540 16946 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.511483 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.512151 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.512153 16946 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.513274 17006 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.513350 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.513953 17006 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.516251 17006 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.516782 17006 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:58.517302 17006 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.517688 17006 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:58.523377 16996 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.523448 16976 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.524539 16996 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.525905 16976 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.526563 16976 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.527086 16976 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.529726 16992 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.530417 16997 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.530381 16992 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.576686 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.445420 17000 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.482244 16994 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.531005 16997 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.587172 16976 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.527539 16996 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:02:58.591375 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78880.6us
I20260709 13:02:58.592913 15084 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:02:58.611608 15087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:02:58.598649 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.610673 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.540083 17006 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.599628 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.623526 15087 raft_consensus.cc:606] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:58.608448 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.624286 15847 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:58.591080 16980 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.633877 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.610711 17012 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.636374 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.599601 17003 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.637835 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.624233 17001 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.636507 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.638813 17005 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.640234 15843 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:58.619463 17009 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.608033 17004 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.619675 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.634227 17011 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.643688 16997 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.642613 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:02:58.641978 17006 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:02:58.640288 17005 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.637174 16994 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.645744 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:58.630447 16996 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.649801 16977 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.649943 17010 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.645841 17007 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.651575 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.656862 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.650894 16992 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.653087 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.645846 17009 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.654875 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.654630 16996 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.655947 17007 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.592201 17000 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:58.705065 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53874.2us
I20260709 13:02:58.702677 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.658519 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.710839 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.699278 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.730909 15082 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:58.735371 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.732028 15081 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:58.700068 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.652422 16976 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.651254 17004 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.652961 16977 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.743360 15086 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:58.747430 15844 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:58.699501 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.643877 17011 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.755383 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.758658 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.642937 17012 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.760810 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.659951 17010 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.646409 17003 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.660794 17002 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.744603 16976 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.764536 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.769150 16980 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:58.769369 16935 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.770227 16996 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.771399 17004 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.773924 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.782833 15847 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:02:58.784314 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.785091 16996 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.785271 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.785494 16996 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.785732 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.786612 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.785974 15079 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:02:58.785907 16996 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.644621 17001 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.777149 16980 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.787696 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:58.787034 16935 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.787276 16992 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.788443 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.791155 16977 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.816474 16992 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.817080 16992 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.817283 17009 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.817981 16980 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.783843 16946 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:58.785799 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800067a20 after lost signal to thread 13970
I20260709 13:02:58.820170 15083 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.822091 16988 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.822757 16946 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:58.826107 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.824131 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.828711 16988 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.829017 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.829191 16988 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.830997 16823 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.831049 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.833067 15229 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:58.788976 17007 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.831559 16823 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.791321 17012 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.844317 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.832886 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.839149 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.840374 16998 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.835974 15843 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:02:58.841424 16976 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.859690 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.843299 16823 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.853682 16986 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.853852 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.834640 16988 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:58.854969 17005 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.845001 15229 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:58.855489 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.855917 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.860802 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.865962 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.866814 15844 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:58.868764 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.865497 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.875003 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.868708 16982 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:58.876549 16982 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.876991 16982 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:58.878170 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.877615 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:58.864527 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800091100 after lost signal to thread 15020
I20260709 13:02:58.881173 16975 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.884722 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.884893 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.885432 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.885771 15235 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:02:58.881827 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.887454 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.887885 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.889822 17003 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.890254 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.874002 16997 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:58.873656 16989 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:58.891376 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.892247 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.896732 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.898334 15083 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:58.898458 16823 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:58.898890 16823 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.881472 16998 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.894229 16990 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:58.906396 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.897270 17004 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:58.893013 16997 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.881776 16975 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.910473 16823 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:58.909384 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:58.908597 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800322a80 after lost signal to thread 15112
I20260709 13:02:58.909921 16997 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:58.917774 16973 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.906469 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.907577 16990 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:58.924319 17017 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.913815 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.906445 16974 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.925532 16973 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.927160 16971 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:58.928174 17001 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.930264 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.908828 17004 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.950521 17004 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:58.950588 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.947721 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.911478 17019 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.933723 16990 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.938755 17017 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.940174 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.940342 16996 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.941260 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.941339 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.882836 16986 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.942461 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:02:58.944278 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:58.965437 15083 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:58.945223 16957 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.945580 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.945258 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:02:58.947264 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:58.945336 16974 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:58.907127 17000 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:58.958289 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:58.974565 15082 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:58.975215 15017 debug-util.cc:398] Leaking SignalData structure 0x7b08002d41c0 after lost signal to thread 15187
I20260709 13:02:58.972416 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:02:58.976333 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.976995 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.977571 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.979621 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.981385 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.981515 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.985786 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:58.986578 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.990609 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:58.987408 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:58.995613 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:58.996618 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.015190 15845 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
W20260709 13:02:59.002575 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 103810us
I20260709 13:02:59.010959 16823 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:59.018496 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.018236 16823 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.018417 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.019599 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.021353 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.020630 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.019505 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.024909 16975 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.025606 17020 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:02:59.004873 15017 debug-util.cc:398] Leaking SignalData structure 0x7b080037ed60 after lost signal to thread 15188
I20260709 13:02:59.029107 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.030946 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:02:59.035966 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50262.6us
I20260709 13:02:59.038627 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.000957 17000 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:59.039592 15162 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:59.037871 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.057118 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.058789 17023 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:59.059661 17023 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.019853 16987 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.014175 17003 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:58.992978 16982 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.004066 16935 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.078256 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.086200 16823 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.087972 16823 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:59.035640 16976 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.078956 17002 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.079559 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.017977 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.029786 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.105427 15237 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 2: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:02:59.070596 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.097585 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:02:59.098027 13972 debug-util.cc:398] Leaking SignalData structure 0x7b080000f9c0 after lost signal to thread 15127
I20260709 13:02:59.031493 15843 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:58.960984 16971 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:58.973212 17019 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.102643 17023 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.095476 16979 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.103499 16823 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.075304 16987 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.134307 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.104008 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:59.042537 17000 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.072476 17007 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:59.104686 16978 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.105832 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.078024 16988 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:02:59.075688 16946 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.119052 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.125968 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.132711 15082 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:02:59.137634 16973 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.126724 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.139336 16823 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.109087 17002 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:02:59.125502 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800372520 after lost signal to thread 15261
I20260709 13:02:59.093851 17003 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.160748 15844 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:02:59.160781 15155 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:59.141299 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.165542 16823 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:59.174342 16823 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.175804 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:59.177609 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 71856.4us
I20260709 13:02:59.142118 16987 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:59.126762 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.026315 17020 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.125339 15236 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:02:59.161566 16976 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:59.168936 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.169412 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.200119 14983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 362) took 2243 ms. Trace:
I20260709 13:02:59.200181 16976 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:59.200697 14983 rpcz_store.cc:276] 0709 13:02:56.943318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.381107 (+1437789us) service_pool.cc:224] Handling call
0709 13:02:59.186150 (+805043us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260709 13:02:59.200297 15154 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:02:59.158679 17017 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:02:59.169591 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149043us
I20260709 13:02:59.205796 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.171015 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.181107 15228 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:59.142553 17000 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.167852 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:59.018206 16957 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.184379 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.212337 16823 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:02:59.212968 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45261.1us
I20260709 13:02:59.207684 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:02:59.164801 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 140904us
I20260709 13:02:59.105139 17018 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:59.165113 17002 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:59.141567 16992 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.201318 16976 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.126858 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.210741 16979 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.169448 16996 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.180107 16993 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:59.227203 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.180223 17007 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:59.185621 16989 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.196715 16975 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:59.234449 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:02:59.240828 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 62376.2us
I20260709 13:02:59.242817 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.236204 17017 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:02:59.229276 16983 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.251307 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.294646 15161 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:02:59.230394 16946 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:59.243683 15017 debug-util.cc:398] Leaking SignalData structure 0x7b08002901c0 after lost signal to thread 15872
W20260709 13:02:59.293473 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63336us
I20260709 13:02:59.298906 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.300346 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.254447 17011 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.301450 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.302585 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.303402 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.303684 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.241655 16971 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.304419 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:59.304207 15159 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:02:59.255887 16989 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.256286 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.306628 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.293081 15085 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:59.298136 16996 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:59.312670 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:59.301595 15017 debug-util.cc:398] Leaking SignalData structure 0x7b080037c9c0 after lost signal to thread 16919
I20260709 13:02:59.314383 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.318027 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.318338 17021 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:02:59.301081 16993 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.324898 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.301110 17011 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.325594 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.330626 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.331822 16971 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:59.309724 16935 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.307734 16946 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:02:59.312338 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.339545 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:59.329337 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800270520 after lost signal to thread 16958
I20260709 13:02:59.296150 16975 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.315798 16996 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:59.341845 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.343050 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.248564 17018 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.350574 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.332634 16971 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:02:59.343255 15017 debug-util.cc:398] Leaking SignalData structure 0x7b080032e360 after lost signal to thread 16963
I20260709 13:02:59.407976 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.354815 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:02:59.388599 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800384940 after lost signal to thread 16964
I20260709 13:02:59.346752 16996 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.541872 15163 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
I20260709 13:02:59.408433 17016 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.339748 16974 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.342900 16986 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.409659 17026 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.349572 16983 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:59.347813 17007 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.350893 16988 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:02:59.400949 17003 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.345623 17004 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.553763 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.554883 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:59.548956 16980 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.730s	user 0.021s	sys 0.010s
W20260709 13:02:59.550724 15017 debug-util.cc:398] Leaking SignalData structure 0x7b08003027a0 after lost signal to thread 16965
I20260709 13:02:59.562465 16984 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:59.550945 16997 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.617s	user 0.021s	sys 0.000s
I20260709 13:02:59.575426 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.580766 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.525252 17019 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.589754 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:59.567476 16992 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.597195 16975 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.344146 17021 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.599221 17020 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.601527 17030 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:59.599448 17016 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.579861 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:02:59.640204 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.641098 17030 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.641390 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.641692 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:02:59.642343 15232 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:02:59.643781 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:02:59.645012 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64314us
I20260709 13:02:59.638335 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.641664 17030 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.646061 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.581336 16997 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.650305 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:59.558679 16946 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:59.638091 17033 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:59.650785 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.579022 16980 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.651328 17016 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.665205 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.657972 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:59.692196 17001 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:59.658191 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 327632us
I20260709 13:02:59.702179 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.703434 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.658620 17016 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.703730 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.703800 17008 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.703758 17004 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.716797 17016 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:59.722640 16983 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.733103 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.710842 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.733228 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:59.718631 15234 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:59.706157 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:02:59.758014 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52060.9us
I20260709 13:02:59.758060 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:59.758785 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.759857 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.760933 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.765694 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.757197 17008 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.770412 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:59.759114 16983 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.760205 16971 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.764209 16981 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.805217 17029 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.763314 16979 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.806074 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.806600 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.772289 16983 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:59.766423 17026 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:59.809473 16997 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.776120 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:02:59.839466 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 85200.6us
I20260709 13:02:59.817788 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:02:59.841295 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:02:59.770779 17008 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:59.837970 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:02:59.867120 15235 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:59.769191 17018 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.771865 17037 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.889164 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:02:59.832036 16998 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.874070 17021 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.880070 16997 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:59.874686 17022 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:59.887346 17002 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.643s	user 0.003s	sys 0.016s
W20260709 13:02:59.902168 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68603.3us
I20260709 13:02:59.918406 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:02:59.919420 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:02:59.900424 16997 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.915943 17010 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.925138 16997 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.913569 17002 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.941871 16998 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:02:59.945397 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.006s	sys 0.031s
W20260709 13:02:59.946878 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.006s	sys 0.032s
W20260709 13:02:59.947299 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 641368us
W20260709 13:02:59.948266 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 187409us
I20260709 13:02:59.928285 17036 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.949656 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:02:59.950783 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:02:59.951515 17029 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.952128 17029 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:59.952441 17029 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.952788 17029 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:59.955008 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 89529.5us
I20260709 13:02:59.958166 17021 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:02:59.958761 17021 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:02:59.936384 17000 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.725s	user 0.007s	sys 0.007s
I20260709 13:02:59.964780 17002 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:02:59.966390 15844 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:59.971403 16989 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.646s	user 0.005s	sys 0.014s
I20260709 13:02:59.971846 16989 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:02:59.948295 16998 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:02:59.974680 16998 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.927135 17041 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:59.977262 17041 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:02:59.977707 17041 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.979775 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:02:59.983852 16987 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.780s	user 0.005s	sys 0.013s
W20260709 13:02:59.962628 16990 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.003s	user 0.009s	sys 0.004s
I20260709 13:02:59.984305 16987 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.006389 17034 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.007858 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:02:59.960829 17021 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:02:59.982518 17000 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.021575 15847 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:00.022625 17000 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.023686 15083 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.024626 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.005362 16988 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:02:59.955632 16996 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.025990 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.026247 16988 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.027562 16996 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.027933 15086 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.028210 15084 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.028321 16988 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.029094 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.029417 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.029850 15082 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:00.030905 16982 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.031400 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.033794 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.034711 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.036216 15079 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.037160 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.037804 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.038853 15082 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.039453 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.016350 16994 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:00.013765 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 94289.6us
I20260709 13:03:00.014005 16990 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.018653 17035 raft_consensus.cc:993] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:03:00.041122 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.041388 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.044497 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:00.045778 15231 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.048175 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.049335 16990 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.049655 16982 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:00.050107 16982 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.050294 16823 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.050565 16982 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.050741 16823 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.051613 16823 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.019191 16997 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.054276 15229 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.054816 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:00.055024 16823 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.054877 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.057861 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.058622 17026 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.062282 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:00.064625 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 174503us
I20260709 13:03:00.066414 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.053673 17036 raft_consensus.cc:1081] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:03:00.069430 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.052524 16983 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.071800 15087 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.050384 16977 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:03:00.074329 16977 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.074771 16977 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:00.075644 16977 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.076660 16977 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.071283 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.018136 17030 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.065124 16988 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:00.066320 17026 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:00.081743 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.083984 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1515) took 1056 ms. Trace:
I20260709 13:03:00.084370 15228 rpcz_store.cc:276] 0709 13:02:59.027697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.139824 (+112127us) service_pool.cc:224] Handling call
0709 13:02:59.140119 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.210069 (+ 69950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.083741 (+873672us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400295254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:00.083920 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93783424}
I20260709 13:03:00.079468 17024 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.101030 16997 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.101874 17016 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.109193 17008 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.082139 16988 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.111837 16988 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.105556 16989 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.117158 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2196) took 1152 ms. Trace:
I20260709 13:03:00.117607 15163 rpcz_store.cc:276] 0709 13:02:58.965058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.965174 (+   116us) service_pool.cc:224] Handling call
0709 13:03:00.116934 (+1151760us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540013f6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:00.117057 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808994560}
I20260709 13:03:00.119562 16994 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:00.077190 17010 consensus_queue.cc:1175] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 13:03:00.107106 17030 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:00.090579 17026 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.072402 17014 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.125765 15081 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:00.121227 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47494.1us
I20260709 13:03:00.122583 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.106519 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:00.120977 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56653.3us
I20260709 13:03:00.130046 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.132790 16983 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.127473 17026 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.132539 17041 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.137537 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:00.139961 17041 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.145542 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.150239 16987 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.153357 17029 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.157766 16957 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.159920 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.160441 17029 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.161803 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:00.162925 15847 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:00.164126 17016 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:00.163618 14985 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.166006 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.167402 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:00.171449 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.172289 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1643) took 1279 ms. Trace:
I20260709 13:03:00.172690 15229 rpcz_store.cc:276] 0709 13:02:58.892368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.892507 (+   139us) service_pool.cc:224] Handling call
0709 13:03:00.172086 (+1279579us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400319054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:00.172203 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2640216832}
I20260709 13:03:00.155090 16980 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.174309 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1511) took 1148 ms. Trace:
I20260709 13:03:00.174454 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.175217 14985 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 4, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.176079 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.178773 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.179276 16823 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:00.174712 15231 rpcz_store.cc:276] 0709 13:02:59.025875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.025992 (+   117us) service_pool.cc:224] Handling call
0709 13:02:59.026262 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.046210 (+1019948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.174061 (+127851us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400175a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:00.174219 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344309120}
I20260709 13:03:00.185997 15839 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.186518 15842 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.178109 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.188477 15845 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.188966 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.189568 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.190141 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.191824 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.193562 16823 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:00.194024 16823 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.194857 16823 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.196017 16823 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:00.197764 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 130326us
I20260709 13:03:00.201607 17008 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:00.201512 17021 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:00.203436 17018 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:00.205299 17018 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.205636 17018 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.206462 17018 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.208088 15128 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.208366 17020 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:00.204283 16971 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:00.209748 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.211591 15839 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.211514 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.212950 16980 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.217569 15839 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.217905 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:00.216802 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.221868 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:00.220206 15814 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.624s (SENT) [suppressed 1 similar messages]
I20260709 13:03:00.224285 15081 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.225382 17030 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.225673 17019 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.229271 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.230059 16982 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.231021 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.232365 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.233711 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.235140 17018 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.235261 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.236953 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.237710 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:00.238054 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.242697 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.246308 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.249396 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.250218 15229 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 2: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.250569 15231 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.252058 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.254647 15084 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.254429 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.255654 15083 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.257999 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.260313 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.266407 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.267706 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.268821 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.256534 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.273123 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.275226 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.221310 16987 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.279112 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.281337 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.232699 16982 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.283741 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.284610 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.243515 16972 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.243637 16980 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.288771 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.289371 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.290133 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.250599 17019 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.246143 17042 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:00.309263 17042 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.249418 17004 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.290815 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.247175 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.299057 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:00.316243 15081 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:00.302876 16988 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.287782 16972 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.309747 17042 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:00.332764 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.337236 15163 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:00.338485 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:00.338660 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 101794us
I20260709 13:03:00.340446 15231 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.292528 16984 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:00.350499 15082 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:00.340401 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.343046 15086 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:00.317952 16971 consensus_queue.cc:1243] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:00.345671 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:00.346882 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.344826 16972 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:00.354827 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 134645us
I20260709 13:03:00.360879 15159 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 2: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.352181 17021 consensus_queue.cc:1243] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:00.363761 15163 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:00.375133 15236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "211046e319804173a9b8954112b8d044"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:03:00.375932 15236 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:00.376937 15236 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.379473 17026 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:00.370231 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149238us
I20260709 13:03:00.362831 17007 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:03:00.382962 15082 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:00.384686 15160 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:00.387033 15160 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.339604 16987 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:03:00.392172 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.397315 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:00.400492 17052 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:03:00.402971 15236 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.363224 16971 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:03:00.403306 15160 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.361403 17010 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:03:00.422180 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.404901 16972 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.405279 16957 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:00.412639 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.404232 17032 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.386404 16988 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.416628 15160 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.427951 15229 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.417771 17049 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.419250 17002 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:00.432582 15235 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.437134 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.434016 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.435436 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.438679 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.443167 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:00.445989 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.451429 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:00.453701 15229 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.408668 17021 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:03:00.458762 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:00.457566 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45012.1us
I20260709 13:03:00.429394 17032 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.455564 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.387220 17048 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:03:00.444011 16999 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.467190 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.470393 15235 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:00.444911 17009 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.481046 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.467620 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.478063 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.478190 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.474653 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.478842 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.481096 15160 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:00.484221 15228 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.484457 15082 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:00.485013 15082 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:00.485482 16823 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:03:00.461789 17007 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:00.486534 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:00.486627 17007 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.487026 16957 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:00.487286 17007 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.487425 16957 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.488360 16957 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.465338 17045 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.489550 16957 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:00.457026 17046 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:00.464174 17026 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.486755 17048 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.474546 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.496549 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.496913 15233 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.497056 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.497774 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.498032 15233 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.493075 15232 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:00.452939 15236 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.497965 17045 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:00.491541 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.490847 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.525194 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.526765 17021 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:00.521387 15082 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.528280 15082 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.533082 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:00.470993 17050 raft_consensus.cc:1064] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:03:00.499680 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.519737 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:00.523212 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 61917.8us
I20260709 13:03:00.540280 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.542495 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.548926 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.520776 16994 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.551117 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.553174 16984 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:00.557379 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.555680 17028 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:03:00.560566 17028 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.553687 16989 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.561116 17028 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:00.556666 17045 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.538623 17046 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:00.568303 17046 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:00.569303 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44192.6us
I20260709 13:03:00.570245 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.570963 17052 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.571990 17048 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.572774 17048 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.574059 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.577649 17018 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.583545 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.584350 15155 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.586145 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.556859 17014 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.592775 15087 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.593396 16989 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:00.595343 16989 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:00.596418 16989 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:00.598105 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48481.6us
I20260709 13:03:00.601246 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.602177 17014 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.604072 15158 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:00.604379 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.548128 16987 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.611104 15082 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:00.527379 17021 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.612414 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:00.611994 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 56055.4us
I20260709 13:03:00.615660 15155 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:00.622473 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:00.621922 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 81775.4us
I20260709 13:03:00.630246 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.610633 17035 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.621261 16972 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.618767 15840 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.620640 17025 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:03:00.602818 17028 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.634038 16982 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.610555 17024 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:03:00.610302 16984 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:03:00.694325 15846 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:00.633615 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.639703 17054 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:03:00.704067 16972 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:00.704785 17058 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.705250 17058 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.705397 16972 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:00.705778 16972 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.706609 17021 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.706198 17058 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.706730 16972 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.707695 17058 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:00.752272 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 116528us
I20260709 13:03:00.753542 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:00.634243 16987 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:00.755931 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 122421us
W20260709 13:03:00.756978 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 305579us
I20260709 13:03:00.758121 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:00.761401 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.766772 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.773093 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.776897 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.736239 17004 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:00.782385 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.763309 17054 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:03:00.708066 17045 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:03:00.788276 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 167371us
I20260709 13:03:00.762681 16972 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:00.768235 17052 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.792477 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.781769 17021 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:00.768692 17009 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.801297 17021 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.772497 16987 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:00.804826 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.805085 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:00.808940 16984 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.777601 17002 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.777153 17000 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.825069 17030 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.825613 17030 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:00.826591 17030 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.734557 17042 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.803889 17028 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.842720 15237 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:00.846537 15082 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:00.847291 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:00.849810 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.843493 16999 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.856031 16823 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:00.856621 16823 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.858515 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.803210 16987 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.843591 17009 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:00.806551 17021 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.862897 16823 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:00.846451 17002 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.855696 16982 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:00.875826 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.868476 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.880609 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:00.867429 17008 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.866689 17016 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.864800 16946 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:00.886577 16971 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.878548 17048 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:03:00.859685 17007 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.910750 17042 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:00.911729 17057 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:03:00.912066 17007 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:00.864311 16999 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:00.917104 17059 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:03:00.915251 16946 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.918124 16999 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:00.919226 16946 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:00.904453 17002 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.920928 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:00.940789 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:00.912776 17007 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:00.983353 17032 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:00.988075 17048 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:00.988517 17048 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:00.989296 17048 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:00.990458 17048 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.988700 17032 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:01.014458 15231 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.923695 17016 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.036386 16994 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:01.036929 16994 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:01.037760 16994 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:01.039642 16983 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:01.040110 16983 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:01.041133 16983 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:01.050851 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.066176 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 194033us
I20260709 13:03:01.042685 16983 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:01.073802 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:01.078588 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 210887us
I20260709 13:03:01.103322 15083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
W20260709 13:03:01.108330 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 249925us
I20260709 13:03:01.109624 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.112883 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.116811 17052 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:01.115347 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.120232 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.131234 17071 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:01.131553 16990 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:01.131681 17049 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:01.132225 17049 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:01.117317 17052 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:01.131858 17071 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.117842 16972 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.136096 16990 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:01.126863 16971 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:01.129747 17018 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.155221 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.158565 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.163291 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.169625 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.173735 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:01.177424 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.178318 17069 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.178395 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:01.179657 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:01.183106 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:01.181267 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:01.184129 15229 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:01.160599 17020 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:01.168478 16975 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.190924 15157 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:01.194813 17068 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.192600 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.186806 16980 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:01.224113 15236 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:01.224556 17069 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.226634 16981 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:01.227582 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.173170 17018 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.173939 17016 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:01.263091 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 382595us
I20260709 13:03:01.271314 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.347986 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:01.350617 15839 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:01.351740 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:01.229197 17069 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:01.354800 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.153266 16990 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.172518 17052 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:01.269759 17068 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.273594 17053 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:01.384961 17053 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:01.282871 17020 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:03:01.392611 15844 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:01.392125 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.390278 16990 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.391927 15154 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:01.365001 17075 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:01.416353 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.418291 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.423098 15237 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:01.434762 15081 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:01.473943 16988 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:01.487967 15161 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:01.471992 17000 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.526844 15080 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:01.500823 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:01.538121 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:01.539455 17046 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:01.539953 17046 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.540814 16981 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:01.542605 16981 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.546025 17046 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.548364 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.549377 15080 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:01.530354 17071 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:01.554440 17030 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:01.516690 16984 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:03:01.555444 16972 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.500366 17053 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:01.583758 16972 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:01.607008 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.585037 16975 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:03:01.634454 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 363302us
W20260709 13:03:01.640468 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.011s	sys 0.009s
W20260709 13:03:01.641888 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.011s	sys 0.009s
I20260709 13:03:01.626636 16946 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:01.642534 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.643054 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 526189us
W20260709 13:03:01.644470 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.644495 17080 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:01.649142 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 128934us
W20260709 13:03:01.652084 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.012s	sys 0.011s
W20260709 13:03:01.652576 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.012s	sys 0.011s
I20260709 13:03:01.642995 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:01.653589 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 536317us
I20260709 13:03:01.661408 17091 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:01.665048 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.665000 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:01.666777 16979 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:01.613135 17049 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.669893 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.674569 15081 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:01.667352 16979 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.671447 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.650977 16946 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.653371 17087 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:01.683355 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.690328 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:01.694811 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.697131 16989 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:01.701186 15847 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:01.708640 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65728.8us
I20260709 13:03:01.710116 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:01.713083 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:01.726227 15848 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:01.720909 17002 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:03:01.728634 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.730384 17000 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.676115 17056 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:03:01.733058 17082 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:03:01.740079 17048 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.001s
I20260709 13:03:01.719368 16977 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:03:01.742556 16983 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.670s	user 0.007s	sys 0.005s
I20260709 13:03:01.724819 17071 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.743577 16979 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.743919 17071 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.746856 17071 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:01.733350 17032 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.001s
I20260709 13:03:01.734169 17092 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:01.748780 17092 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:01.744748 14985 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 259) took 1652 ms. Trace:
I20260709 13:03:01.750900 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:01.754475 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:01.757421 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.759169 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:01.759939 15163 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:01.760977 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:01.761638 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:01.762559 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:01.764951 17092 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:01.766072 15154 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:01.766733 17092 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:01.769948 17075 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.769739 16972 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:01.385326 17058 consensus_queue.cc:1175] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:01.731668 17050 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.772027 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:01.785384 17093 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.787045 17082 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:01.788177 17046 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.790370 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:01.794625 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:01.797338 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:01.803280 15234 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:01.773370 15083 raft_consensus.cc:606] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.807112 15083 raft_consensus.cc:614] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:01.807870 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:01.808822 15846 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:01.811647 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:01.735369 16946 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:01.821674 17064 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:03:01.801679 17041 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.805757 17075 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.825184 17028 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.705087 17087 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:01.823061 17037 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.841199 17041 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.841396 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:01.841379 17075 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.848136 17008 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:01.848579 16980 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.811540 17046 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:01.859648 17008 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.815932 17093 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.871610 17075 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.872143 17008 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:01.872808 17093 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.867360 16980 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.873909 17041 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.874713 17044 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:03:01.876106 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.835968 17082 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.875245 16980 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:01.879981 16994 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.889278 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:01.890385 17049 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:01.868139 16990 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:01.750224 14985 rpcz_store.cc:276] 0709 13:03:00.092587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.092799 (+   212us) service_pool.cc:224] Handling call
0709 13:03:01.744657 (+1651858us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:00.348923 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:00.366028 (+ 17105us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:00.366089 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:03:00.366109 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:00.366531 (+   422us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:00.366613 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:00.366641 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:00.366976 (+   335us) tablet.cc:701] Row locks acquired
0709 13:03:00.366995 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:03:00.367207 (+   212us) write_op.cc:270] Start()
0709 13:03:00.367327 (+   120us) write_op.cc:276] Timestamp: P: 1783602180367103 usec, L: 0
0709 13:03:00.367351 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:03:00.426273 (+ 58922us) log.cc:844] Serialized 921 byte log entry
0709 13:03:00.803393 (+377120us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b18000e74c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:01.391394 (+588001us) op_driver.cc:464] REPLICATION: finished
0709 13:03:01.658126 (+266732us) write_op.cc:301] APPLY: starting
0709 13:03:01.672441 (+ 14315us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:01.672597 (+   156us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:01.672621 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:01.673042 (+   421us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:01.673215 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:03:01.673257 (+    42us) write_op.cc:312] APPLY: finished
0709 13:03:01.673449 (+   192us) log.cc:844] Serialized 29 byte log entry
0709 13:03:01.673678 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.675871 (+  2193us) write_op.cc:454] Released schema lock
0709 13:03:01.678311 (+  2440us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:01.703395 (+ 25084us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.703446 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39811,"reported_tablets":2,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":255822,"apply.run_cpu_time_us":2263,"apply.run_wall_time_us":49062,"num_ops":2,"prepare.queue_time_us":146265,"prepare.run_cpu_time_us":68717,"prepare.run_wall_time_us":319932,"raft.queue_time_us":476706,"raft.run_cpu_time_us":18612,"raft.run_wall_time_us":243345,"replication_time_us":1023827,"spinlock_wait_cycles":98935680,"thread_start_us":736103,"threads_started":4,"wal-append.queue_time_us":278222}]]}
I20260709 13:03:01.860803 17030 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:03:01.875253 17044 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:01.919469 17046 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:01.856477 17037 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:01.916893 16990 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:01.946493 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.969430 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 295872us
I20260709 13:03:01.971648 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:01.973529 17098 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.974308 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:01.974627 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79029.2us
I20260709 13:03:01.975720 17098 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:01.976188 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:01.979592 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:01.980242 15158 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:01.980515 17042 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:01.992002 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:02.004144 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.006484 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:02.007354 15229 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:02.002907 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:02.015044 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 336063us
I20260709 13:03:02.016469 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:02.018587 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:02.006798 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:02.023373 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.026706 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:02.033692 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55745.4us
I20260709 13:03:02.035559 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:02.038944 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:02.007321 17098 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:01.993702 17044 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.048655 16998 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:02.025245 17080 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:02.049110 16998 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.060030 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:02.060129 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 226241us
I20260709 13:03:02.007584 17007 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:02.063907 15082 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:02.067471 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.034425 17000 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.035094 17060 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.041502 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:02.038255 15236 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:02.073832 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:03:02.045459 17098 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:02.045943 17071 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:02.085201 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.090726 17075 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:02.091545 17075 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:02.094872 17035 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:02.095340 17035 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:02.095966 17075 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:02.087698 17088 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:03:02.087563 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.068982 16998 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:02.075219 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.104002 15079 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:02.088414 17098 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:02.116578 16990 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.106900 17058 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:02.118448 17028 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.139786 17037 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.131106 17058 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.136329 17028 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:02.165915 17053 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:02.145713 17087 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:02.172078 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:02.148501 17103 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.189126 17103 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:02.195060 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 155576us
W20260709 13:03:02.194190 16971 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.039s	user 0.005s	sys 0.011s
I20260709 13:03:02.189612 17058 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.218679 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:02.219673 15161 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:02.214969 16971 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.220420 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:02.247705 17007 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:02.253401 17042 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:02.266897 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.398478 17103 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:02.400247 17049 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.007s
I20260709 13:03:02.403838 17098 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.410086 16998 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:02.417053 17092 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.650s	user 0.016s	sys 0.000s
W20260709 13:03:02.434170 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 142443us
W20260709 13:03:02.454241 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 390272us
I20260709 13:03:02.465992 17052 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:02.449076 17056 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.465349 17058 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.479490 17056 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.479614 17058 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.487920 17099 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.488711 17099 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.491614 17099 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.498497 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:02.500012 15088 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:02.499051 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:02.515707 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.482724 17028 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:02.518858 15081 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:02.526227 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.532573 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:02.516234 17099 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.573381 15053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.014s	sys 0.033s
W20260709 13:03:02.573961 15053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.014s	sys 0.033s
I20260709 13:03:02.528014 17098 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:02.578960 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.575501 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:02.522156 17089 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:02.578154 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.021s	sys 0.017s
W20260709 13:03:02.603026 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.022s	sys 0.017s
I20260709 13:03:02.603847 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:02.611732 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.021s	sys 0.030s
W20260709 13:03:02.612213 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.021s	sys 0.030s
I20260709 13:03:02.639904 17019 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:02.640420 17019 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:02.641374 17019 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.529697 17056 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.675721 17110 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.677022 17070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.003s	sys 0.061s
W20260709 13:03:02.677493 17070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.063s
I20260709 13:03:02.685825 15157 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:02.689818 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:02.695727 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 89990.4us
I20260709 13:03:02.696159 16999 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:02.651788 16981 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.702723 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:03:02.705335 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.002s	sys 0.011s
W20260709 13:03:02.713541 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.002s	sys 0.011s
I20260709 13:03:02.669021 17073 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:02.678300 17089 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.664067 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.678579 17046 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.741654 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.679534 17058 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:02.705689 15160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.037s	sys 0.013s
I20260709 13:03:02.665704 17043 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.722453 14984 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 353) took 2158 ms. Trace:
I20260709 13:03:02.665923 16946 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:02.738327 15845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.024s	sys 0.019s
W20260709 13:03:02.751806 15845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.024s	sys 0.019s
W20260709 13:03:02.751431 15160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.037s	sys 0.013s
W20260709 13:03:02.738286 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.012s	sys 0.031s
I20260709 13:03:02.575196 17099 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.665460 16984 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:02.676378 17110 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:02.742832 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 167449us
W20260709 13:03:02.744592 17044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.005s
I20260709 13:03:02.695740 17107 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.765902 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.012s	sys 0.031s
I20260709 13:03:02.751744 14984 rpcz_store.cc:276] 0709 13:03:00.544212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.544394 (+   182us) service_pool.cc:224] Handling call
0709 13:03:01.044524 (+500130us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:02.702494 (+1657970us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":284480384}
I20260709 13:03:02.721992 17073 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:02.785668 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 721812us
I20260709 13:03:02.782277 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.774782 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:02.782231 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:02.865067 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 76855.7us
W20260709 13:03:02.917905 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.748092 17045 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.770115 17082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.005s	sys 0.017s
I20260709 13:03:02.899375 15086 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:02.930231 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.772710 17043 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:02.772746 16946 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.739074 16981 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:02.941974 17089 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:02.764873 16999 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:02.981218 17120 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.032898 16981 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.033141 16946 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:03.033241 17043 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.033890 17037 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.037289 17111 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.144017 17037 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.039469 17114 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.145232 17114 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:02.959410 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:03.130136 16946 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:03.216715 16946 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:02.773592 17044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.006s	sys 0.006s
I20260709 13:03:03.220499 17118 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.138705 16999 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.773248 16984 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.144117 17111 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:03.047434 17121 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:03:02.826218 17046 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.916369 17058 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.202675 17114 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.204412 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:02.958012 17107 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:02.958339 16983 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:03.147921 17120 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:02.771899 17117 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:02.929571 17073 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:03.238070 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.005s	sys 0.014s
I20260709 13:03:03.221237 17118 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:03.318347 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.006s	sys 0.014s
W20260709 13:03:02.961347 17082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.005s	sys 0.017s
I20260709 13:03:02.917382 17056 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:03.237831 17070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.017s	sys 0.034s
I20260709 13:03:03.312409 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 650) took 1523 ms. Trace:
I20260709 13:03:03.355791 15845 rpcz_store.cc:276] 0709 13:03:01.789321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.799243 (+  9922us) service_pool.cc:224] Handling call
0709 13:03:01.799637 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.800025 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.800068 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.806770 (+  6702us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.207524 (+400754us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b4800260058. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:03:02.752501 (+544977us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:02.752550 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.752649 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.311993 (+559344us) raft_consensus.cc:1685] finished
0709 13:03:03.312019 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.312307 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3548,"raft.queue_time_us":23710,"raft.run_cpu_time_us":4189,"raft.run_wall_time_us":389123,"spinlock_wait_cycles":17240960,"thread_start_us":914286,"threads_started":1,"wal-append.queue_time_us":1123670}
W20260709 13:03:03.253074 15128 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.016s	sys 0.020s
I20260709 13:03:03.277045 16980 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.336054 17110 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:03.358028 16980 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:03.359771 15128 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.016s	sys 0.020s
W20260709 13:03:03.360146 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 757803us
W20260709 13:03:03.360261 17070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.017s	sys 0.034s
I20260709 13:03:03.370100 17070 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.298077 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:03.353322 16983 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.353830 17046 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.355929 17114 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:03.371001 16984 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:03.055327 17045 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.354494 17058 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.370639 17070 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.675800 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:03.508626 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.390456 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2259) took 1639 ms. Trace:
I20260709 13:03:03.722044 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:03.781517 15229 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:03.735932 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:03.723326 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:03.702242 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:03.361591 17056 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.361856 17117 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.585492 16981 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:03.608204 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:04.287907 16971 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.287251 16981 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.758540 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:04.289304 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 680992us
I20260709 13:03:04.297569 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:04.297868 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:03.841032 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.015s
I20260709 13:03:04.290381 15841 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:04.298983 15845 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:04.299474 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.016s	sys 0.018s
I20260709 13:03:04.283712 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.301112 15839 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:04.176792 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:03.708968 17008 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.306322 17008 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.306905 16998 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:03.393353 15085 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:04.308734 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1673) took 1017 ms. Trace:
I20260709 13:03:03.913645 16980 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:04.291795 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 660) took 1517 ms. Trace:
I20260709 13:03:04.309751 17008 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.309839 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 521) took 1057 ms. Trace:
I20260709 13:03:03.903620 17014 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:04.311561 17014 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:03.447540 14983 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:03.841842 15233 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:04.313544 17014 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:03.736549 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:04.315779 15232 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:04.316057 17014 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:03.568949 17026 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:03.397305 17018 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:03.502830 17035 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.317857 17026 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.318167 17018 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.319219 17035 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:04.295328 16979 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.296128 16983 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:03.893762 16946 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:04.309967 15840 rpcz_store.cc:276] 0709 13:03:02.774315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.785487 (+ 11172us) service_pool.cc:224] Handling call
0709 13:03:02.930194 (+144707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.823919 (+893725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.839039 (+ 15120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.842053 (+  3014us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:03.842125 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.291202 (+449077us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:04.291299 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.291593 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976383232}
I20260709 13:03:03.988168 17019 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:04.208774 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 436531us
W20260709 13:03:04.182204 15159 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.005s	sys 0.012s
W20260709 13:03:04.208515 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 451663us
I20260709 13:03:04.291266 15237 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:04.309434 15234 rpcz_store.cc:276] 0709 13:03:03.291373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.735632 (+444259us) service_pool.cc:224] Handling call
0709 13:03:04.308516 (+572884us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:04.270643 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.298884 16981 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:04.172426 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.183794 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2270) took 1263 ms. Trace:
I20260709 13:03:03.843307 17132 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.387567 16979 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.182958 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:03.618470 17041 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:04.390877 15159 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.008s	sys 0.019s
W20260709 13:03:03.759423 17123 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
W20260709 13:03:03.899049 17073 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:03:03.690641 17107 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:03.642496 17052 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.163s	user 0.011s	sys 0.000s
I20260709 13:03:04.310709 15085 rpcz_store.cc:276] 0709 13:03:03.252689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.259546 (+  6857us) service_pool.cc:224] Handling call
0709 13:03:04.309586 (+1050040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.540334 17106 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:03.550017 17110 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:03.976528 17049 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:03.776315 17131 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:03.934123 17141 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:03.843968 17139 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:03.843216 17133 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.380671 16971 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.464774 17091 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:03.898177 17120 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.913204 17138 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.548257 17093 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:04.560424 17070 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:04.525619 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 211165us
I20260709 13:03:04.562961 17144 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:04.532028 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 234738us
I20260709 13:03:04.554304 17149 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:03.599846 17118 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.401541 17111 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.093691 17046 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.984037 17099 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.514400 17026 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:03.709196 17113 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:03.843730 17037 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:03.952059 17142 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.058889 17072 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.550391 16998 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.554311 17053 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.550112 16994 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:04.546689 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.547196 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 871284us
I20260709 13:03:04.560323 17138 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.548621 16946 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.563870 17144 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.580256 17092 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.595922 17141 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.601513 17097 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.523633 17045 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.532708 16979 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.531162 17146 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.565279 17093 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.526577 15156 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:04.590656 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:04.532208 17058 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.618939 17028 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.652985 17093 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:04.613816 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79755.3us
W20260709 13:03:04.625978 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73486us
I20260709 13:03:04.559934 17128 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.651381 17092 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.652401 17145 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:04.657454 16989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.044s	sys 0.078s
I20260709 13:03:04.705710 17058 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:04.705927 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 94816us
W20260709 13:03:04.706291 16989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.045s	sys 0.079s
I20260709 13:03:04.706712 17058 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.707437 17058 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.669975 17138 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.708822 17138 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.709715 17138 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.663884 17110 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:03.792824 17114 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.652292 17146 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.531499 17050 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:04.644982 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 83406.2us
I20260709 13:03:04.530331 17155 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.669124 16946 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.392503 15154 rpcz_store.cc:276] 0709 13:03:02.919757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.949681 (+ 29924us) service_pool.cc:224] Handling call
0709 13:03:03.136587 (+186906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.136918 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:03.137336 (+   418us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.774679 (+637343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.776346 (+  1667us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:03.793352 (+ 17006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.817282 (+ 23930us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.183342 (+366060us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":741,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":117997,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":410844,"threads_started":2,"wal-append.queue_time_us":704358}
I20260709 13:03:04.680225 17041 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.532502 17132 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.658902 16981 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.680255 17075 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.779279 15160 rpcz_store.cc:276] 0709 13:03:01.750973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.751926 (+   953us) service_pool.cc:224] Handling call
0709 13:03:01.752272 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.752704 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.752744 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.794023 (+ 41279us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.764484 (+970461us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:02.764549 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.764651 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.390233 (+625582us) raft_consensus.cc:1685] finished
0709 13:03:03.390252 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.390370 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14167,"raft.queue_time_us":20307,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":933101,"threads_started":1,"wal-append.queue_time_us":1408649}
W20260709 13:03:04.743398 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 162488us
W20260709 13:03:04.715272 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.705811 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80488.4us
I20260709 13:03:04.669589 17091 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.681766 17049 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.728066 17018 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.681798 16998 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.776542 17146 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.776571 17144 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.713107 17139 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.777374 17152 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.737929 17110 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.719905 17053 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.682180 17097 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.681865 17113 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.801481 16946 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.763305 16981 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:04.815164 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47273.4us
I20260709 13:03:04.815778 17052 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:04.803170 17138 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:04.816548 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52779.5us
I20260709 13:03:04.821125 17008 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.739133 17149 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.813887 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:04.805546 17110 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.755635 16994 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.830382 17161 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:04.763162 17073 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.815697 17106 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.811688 17146 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.779244 17075 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.850239 17113 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.770813 17028 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.805354 17145 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.788257 17155 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.872779 17041 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.873095 17146 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.816713 17131 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.862759 17008 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.865100 17099 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.871201 17009 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.872330 17117 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.839248 17161 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.845314 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:04.814060 17128 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.811262 17140 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.142s
I20260709 13:03:04.830276 17142 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.805244 17133 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.872671 17049 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.873652 17110 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:04.892997 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 133636us
I20260709 13:03:04.808625 17050 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.902961 17115 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.833006 17072 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.888005 17161 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:04.906172 17056 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:04.907652 17123 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:03:04.879679 17138 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.903295 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:04.906245 17091 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.918251 17092 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.918402 17028 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:04.918963 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 116852us
W20260709 13:03:04.919087 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 76949.3us
I20260709 13:03:04.919865 16998 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.920486 17026 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:04.921109 17158 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.921350 16994 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.901188 17058 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.929852 17139 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:04.921788 17156 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:04.941880 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:04.942655 15232 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:04.897922 17113 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.943997 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:04.943902 17113 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.946035 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:04.946590 17113 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.948513 16971 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.929255 17114 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.953492 17149 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.953681 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2142) took 2382 ms. Trace:
I20260709 13:03:04.953779 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.954278 15088 rpcz_store.cc:276] 0709 13:03:02.571233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.166444 (+1595211us) service_pool.cc:224] Handling call
0709 13:03:04.167332 (+   888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.560815 (+393483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.564817 (+  4002us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.573157 (+  8340us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:04.573211 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.953283 (+380072us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:04.953338 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.953474 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896243456}
I20260709 13:03:04.938421 17166 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.955873 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.956297 17166 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.946723 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2268) took 2027 ms. Trace:
I20260709 13:03:04.862012 17152 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.949954 17058 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.950137 17139 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.929075 17099 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.951474 17138 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.951701 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.952668 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.952873 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.953223 16981 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.982587 17138 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.953963 17114 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:04.983491 17075 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.984069 16946 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.984182 16994 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.977830 17139 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.984678 16946 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.984920 16994 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.986691 17139 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.986881 16946 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.987254 16994 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.998521 16994 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.998859 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:04.999171 16994 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.999887 15079 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:04.911556 17144 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.952378 17154 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.925616 17009 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.950449 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:04.997428 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1543) took 1265 ms. Trace:
I20260709 13:03:04.977826 16999 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:04.978266 17158 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:04.999171 15839 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:04.978974 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:04.979382 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:04.981079 15841 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:04.983562 17092 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.988739 17050 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.988999 17091 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.989221 17028 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:04.990736 15086 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:05.032019 17091 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.909348 17052 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:05.032984 17052 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.033361 17052 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:05.034307 17091 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.999909 17146 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.991734 17058 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:04.980449 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:04.995781 17163 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.996409 17110 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.997277 15081 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:04.997488 17008 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:04.997707 16998 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:04.924813 17117 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:05.001190 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.957291 15162 rpcz_store.cc:276] 0709 13:03:02.919039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.922355 (+  3316us) service_pool.cc:224] Handling call
0709 13:03:02.922786 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.129344 (+206558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.129390 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.782367 (+652977us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.503426 (+721059us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:04.503501 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.514816 (+ 11315us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.946329 (+431513us) raft_consensus.cc:1685] finished
0709 13:03:04.946349 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.946518 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29238,"raft.queue_time_us":466928,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":358451,"threads_started":2,"wal-append.queue_time_us":697107}
I20260709 13:03:05.018986 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 352) took 2880 ms. Trace:
I20260709 13:03:05.020191 15235 rpcz_store.cc:276] 0709 13:03:03.732303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.732448 (+   145us) service_pool.cc:224] Handling call
0709 13:03:03.732759 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.800423 (+1067664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.800491 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.800588 (+    97us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:04.800650 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.997027 (+196377us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400177354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:04.997090 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.997258 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413153152}
I20260709 13:03:05.025465 15155 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:04.967729 17141 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.975981 17132 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:04.948629 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2272) took 2028 ms. Trace:
I20260709 13:03:05.013343 17155 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.018289 17152 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.032028 17028 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.044384 15840 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:03:05.096973 16998 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.098070 17092 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.049765 16994 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.012286 17115 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.030470 17053 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.050202 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:05.087723 17097 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.098017 17117 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.044333 17166 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.030586 17139 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:05.135980 15160 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:05.135565 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:05.143379 15158 rpcz_store.cc:276] 0709 13:03:02.920444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.923396 (+  2952us) service_pool.cc:224] Handling call
0709 13:03:03.062235 (+138839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.531047 (+1468812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.531095 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.531190 (+    95us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:04.531233 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.948227 (+416994us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:04.948286 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.948465 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3372226944}
I20260709 13:03:05.046187 16999 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.123075 17106 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.176317 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:05.160640 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.156248 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.160928 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.139366 15159 rpcz_store.cc:276] 0709 13:03:02.138559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.138728 (+   169us) service_pool.cc:224] Handling call
0709 13:03:02.140001 (+  1273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.140370 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.140403 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.140908 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.319519 (+1178611us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:03.386610 (+ 67091us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.064793 (+678183us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b180010d6e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:04.555018 (+490225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.555185 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.018600 (+463415us) raft_consensus.cc:1685] finished
0709 13:03:05.018714 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.018910 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16380,"prepare.queue_time_us":66644,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":431253,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":28703,"spinlock_wait_cycles":67644800,"thread_start_us":2285611,"threads_started":2,"wal-append.queue_time_us":1246607}
I20260709 13:03:05.167727 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.168080 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.142731 17163 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.092193 17154 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.118635 17144 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.165266 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:05.161892 17035 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.003s
I20260709 13:03:05.180166 17028 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.219393 17030 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.180037 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.220199 17030 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.180325 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.180593 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.177228 15155 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:05.186452 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.226563 15841 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:05.228932 15839 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:03:05.226152 15232 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:05.186393 15086 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:05.185695 17053 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:05.205642 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 158836us
I20260709 13:03:05.205190 15228 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
W20260709 13:03:05.171761 17014 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.851s	user 0.011s	sys 0.005s
I20260709 13:03:05.166105 17132 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.214818 17158 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.216948 17142 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.218376 17163 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.214077 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:05.222450 17073 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.225314 15154 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:05.240975 17028 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:05.248281 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.249332 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.257519 15162 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:05.242763 17058 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.254084 17053 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.249588 17158 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:05.253336 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:05.242722 17161 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.248421 17092 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.255503 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.212023 17097 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:05.254861 17133 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.260637 15084 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:05.254637 17128 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.260903 17158 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:05.250928 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.233958 16989 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:05.278718 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.279350 15163 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:05.256261 17028 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.280184 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.280799 15163 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:05.281942 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.282519 15163 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:03:05.279815 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.283172 17053 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.284286 17145 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.281409 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.284962 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.285375 15162 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:03:05.281983 15156 raft_consensus.cc:2368] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4.
I20260709 13:03:05.252943 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.286417 15236 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:05.287338 17028 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.276768 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.287992 15080 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:05.288609 17144 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.283370 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.290527 15163 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:05.275759 17019 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.004s
I20260709 13:03:05.270131 17158 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:05.274214 17142 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:05.239651 17139 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.321017 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:05.300603 15202 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.519s (SENT) [suppressed 9 similar messages]
I20260709 13:03:05.275211 17132 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.334872 15159 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:03:05.321712 17073 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.337790 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.329391 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.335731 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.293355 17035 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:05.338652 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:05.323348 17142 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.341622 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.342433 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.319465 17072 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.321810 17168 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.346530 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.346596 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.346889 17169 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:05.289813 17097 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.344043 17009 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.344600 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.347965 15163 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:05.362253 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.348484 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.341142 16971 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.341035 17110 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.349421 17142 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:05.365954 15084 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:05.340827 17152 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.361903 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.367609 17168 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.368245 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.368726 17161 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.344933 17073 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.369218 17169 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.369527 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.371989 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.466190 17052 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.692065 17052 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:05.438871 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65206.3us
I20260709 13:03:05.365528 15080 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:05.439040 15844 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:05.426579 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 99480.9us
I20260709 13:03:05.694640 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:05.387707 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 87114.6us
I20260709 13:03:05.373072 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.499778 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.471096 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.469971 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.689567 15087 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:05.718271 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.414466 15088 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:05.470434 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.395610 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.729089 15160 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 1.
W20260709 13:03:05.434439 17093 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.004s
W20260709 13:03:05.732252 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.437942 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.529354 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.731559 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.739342 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:05.737179 15086 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:05.438398 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.440889 17139 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.740091 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.747469 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.738729 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.426578 17115 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.750219 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.435274 16999 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.752206 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.473492 17117 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.465386 17099 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.748745 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.698990 17030 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.755436 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.438910 17158 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.753043 15080 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:05.764698 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.703286 17131 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.750945 15088 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:05.756775 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.733116 17053 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:05.703315 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.756332 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.366102 17035 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.771899 15228 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:05.698874 17145 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.761827 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.784192 15162 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:03:05.762866 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.784956 15163 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:05.753786 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:05.766165 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.762336 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.765372 15845 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:05.726727 15079 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:05.393735 16971 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.745388 15839 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:05.780023 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.762274 17139 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:03:05.772819 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63497.6us
I20260709 13:03:05.782620 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.744470 17093 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:05.792886 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.793337 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.793519 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.793974 15080 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:05.786808 15235 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:05.782969 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.784289 15086 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:05.702998 17132 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.780766 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:05.787496 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.785319 15236 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:05.790292 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.770620 17053 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.796105 15088 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:05.791891 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.792357 15237 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:05.795428 15846 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:05.753608 17110 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.838406 15083 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:05.842708 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.759538 17152 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.847756 15844 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:05.848687 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.784801 15160 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:05.853489 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.791952 17019 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:05.860901 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:05.866099 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100140us
I20260709 13:03:05.866894 15081 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:05.791504 17139 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.870165 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:05.859386 15154 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.000s
I20260709 13:03:05.851965 17053 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.872656 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.869884 17170 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:05.876693 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.879137 15163 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:03:05.879978 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.882107 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.865389 17019 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.858242 17140 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.882632 15160 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 1.
I20260709 13:03:05.886684 15232 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:05.889089 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.873014 17139 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.791296 17009 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:03:05.892936 17072 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:05.893157 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.921417 15233 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:05.923087 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.925871 17168 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.927604 17073 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.892203 15156 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:05.929416 17169 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.930372 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.932205 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.934096 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.935060 15161 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:05.935990 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.937227 17072 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.937907 17072 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.903911 17097 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.888334 17019 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:05.945263 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:05.946388 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47682.7us
I20260709 13:03:05.843169 17145 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.902691 15084 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:05.880810 17170 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.878497 15154 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:05.907362 17035 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.908311 17152 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.951570 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.880645 15842 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:03:05.909096 17140 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:05.910699 17158 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.914767 17167 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:03:05.943487 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 206750us
I20260709 13:03:05.944361 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.932289 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:05.936704 15161 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:05.949862 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59297.3us
I20260709 13:03:05.950809 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.957396 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:05.956975 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 188166us
I20260709 13:03:05.952494 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.957653 15158 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:05.951145 17170 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:05.970587 15228 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:05.970760 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.972213 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.887346 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:05.974010 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.969265 15160 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
W20260709 13:03:05.977445 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.955148 17097 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:05.977964 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.964797 17167 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:05.964504 17110 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.981412 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.982604 17152 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:05.985885 15233 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:05.979894 17035 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:05.984225 17030 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:05.993651 17035 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.994077 17030 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:05.994213 17035 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:05.994295 17154 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.994603 17030 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.987430 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:05.978996 17073 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:03:05.994688 17140 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:05.985316 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.997686 15228 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:05.992532 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.004995 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.001359 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.012233 15088 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:05.986506 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:05.991811 17099 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:06.013408 17168 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:06.013622 17099 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.991873 17093 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:05.986600 17167 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.014390 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:06.014868 17110 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.015015 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1666) took 1071 ms. Trace:
I20260709 13:03:06.015558 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:05.986359 17152 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.016033 17099 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.015677 15154 rpcz_store.cc:276] 0709 13:03:04.943550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.943687 (+   137us) service_pool.cc:224] Handling call
0709 13:03:06.014738 (+1071051us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540037f854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:06.014924 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639462144}
I20260709 13:03:06.016656 17152 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.015587 17110 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.017706 17110 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.018138 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.018891 15230 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:06.012181 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.009907 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.015057 17167 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.020681 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:05.981865 17097 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:06.011026 15814 leader_election.cc:400] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4.
I20260709 13:03:06.016558 15157 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:06.010001 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.021843 15814 leader_election.cc:403] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:06.021911 15161 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:06.022292 15160 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:06.004909 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.038136 17093 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:06.038720 15840 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 1.
I20260709 13:03:06.038889 17093 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.006237 17145 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.039419 17093 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.039446 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.041358 15233 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:06.043431 17175 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:03:06.038246 16946 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.053347 16946 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:06.053800 16946 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.055830 17053 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.023685 17097 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.060735 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:06.063062 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.064287 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.059219 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.013943 17168 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.069739 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.070317 15163 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:06.071381 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.068289 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.069005 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.085130 15230 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:06.072154 17169 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.086277 17169 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:06.086501 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.086848 17169 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.087379 17106 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.087801 17106 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.088145 17106 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:06.074139 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50463us
W20260709 13:03:06.094604 15202 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.563s (SENT)
I20260709 13:03:06.094722 17139 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.067615 17177 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.066308 15156 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:06.095811 17177 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:06.096356 17177 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.066377 17073 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.098209 17073 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.098695 17073 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:05.989557 17142 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.058511 17117 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.100713 17117 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:06.101213 17117 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.066752 17008 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:06.077114 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.079107 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.103492 15163 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:06.076664 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.079584 17168 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:06.079583 17072 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.081275 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.082445 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 367) took 1102 ms. Trace:
I20260709 13:03:06.112066 15157 rpcz_store.cc:276] 0709 13:03:04.980233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.983047 (+  2814us) service_pool.cc:224] Handling call
0709 13:03:06.016275 (+1033228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.023156 (+  6881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.082238 (+ 59082us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540037f850. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:06.082381 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375472384}
I20260709 13:03:06.082911 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:06.082940 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.095162 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:06.074680 17114 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.103894 15237 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:06.110714 17008 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:06.128124 16946 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.128750 17053 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.130630 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.131085 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.132782 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:06.133731 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50894.9us
I20260709 13:03:06.134320 17145 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:06.102483 17142 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.104701 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.135259 17114 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:06.137331 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 155968us
I20260709 13:03:06.138111 17113 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.138695 16998 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.142263 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1671) took 1040 ms. Trace:
I20260709 13:03:06.278285 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:06.247193 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 152610us
I20260709 13:03:06.289696 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2277) took 1048 ms. Trace:
I20260709 13:03:06.290328 15159 rpcz_store.cc:276] 0709 13:03:05.241811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.255282 (+ 13471us) service_pool.cc:224] Handling call
0709 13:03:06.282860 (+1027578us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b540013e7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:06.289606 (+  6746us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156966784}
I20260709 13:03:06.147667 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.293020 15127 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.299638 17154 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.300163 17154 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.276479 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:06.309639 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.238941 15848 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:06.310640 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.310181 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.309607 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.313591 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:06.313798 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:06.312253 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 240839us
I20260709 13:03:06.318678 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.272878 15155 rpcz_store.cc:276] 0709 13:03:05.102132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.232178 (+130046us) service_pool.cc:224] Handling call
0709 13:03:05.232596 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.351303 (+118707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.351334 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.351381 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.351436 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.141983 (+790547us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540017f054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:06.142051 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.142191 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246706560}
I20260709 13:03:06.314050 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.135605 17008 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.227850 15154 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:06.328325 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.291150 17114 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.334371 15086 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:06.153800 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:06.333143 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56756.2us
I20260709 13:03:06.233343 16971 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.336189 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.232722 17072 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.187244 17110 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.337241 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.341245 15845 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:06.336928 17158 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.342340 17110 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.342767 17158 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.343231 17158 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.343907 17142 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:06.344765 17142 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.345132 17142 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.345162 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.346130 15846 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:06.337569 16946 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.236249 17099 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.313498 17053 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:06.348038 17053 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.347838 17117 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.348613 17053 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.349107 17180 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:06.349442 17099 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.309544 17154 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.310760 16998 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.334640 17008 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.229961 17152 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.227511 17073 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.230522 17035 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.411890 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.413002 15085 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:06.397073 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 292425us
I20260709 13:03:06.394820 15843 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:06.403717 17030 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.414000 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.415148 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.415390 17030 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.416332 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.407653 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.408466 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.437084 15081 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:06.437263 15084 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:06.409683 15839 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:06.410498 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.410624 17073 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:06.416410 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.417255 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.417937 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.418612 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.420059 17180 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:06.407338 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.392150 17132 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:06.424796 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.428907 17142 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.438817 15846 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:06.434019 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:06.445144 15080 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:06.425625 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.396519 16998 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.494974 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.457547 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.425936 17139 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:06.498060 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 175797us
I20260709 13:03:06.498981 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:06.501260 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.492913 15086 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:06.501966 15081 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:06.496280 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 43497.8us
I20260709 13:03:06.427927 17117 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:06.500686 17072 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:03:06.504148 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.502919 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:06.508220 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.455204 17030 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:06.454741 15842 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:06.522343 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 96977.4us
W20260709 13:03:06.453956 17019 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.012s
W20260709 13:03:06.511535 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77639us
I20260709 13:03:06.522404 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.497478 16998 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:06.452546 17180 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:06.504990 17171 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:06.526232 15080 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:06.496137 17132 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.493883 15079 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:06.540856 15845 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:06.532716 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.533078 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.543310 15088 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:06.545267 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.526408 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:06.492146 17093 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.534273 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.503209 17115 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.525590 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:06.943938 15082 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:06.939088 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 392649us
I20260709 13:03:06.533691 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.519940 17030 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:06.558744 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.563680 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.559572 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.560724 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.501874 17097 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.539227 17180 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.636397 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.634521 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.510501 17177 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:06.637848 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.544111 17132 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:06.638616 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 112263us
I20260709 13:03:06.537384 17019 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:06.664705 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 233853us
I20260709 13:03:06.619854 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:06.718617 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:06.729525 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.974902 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:06.848752 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 399104us
W20260709 13:03:06.934635 17113 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.620s	user 0.027s	sys 0.010s
I20260709 13:03:06.535070 17187 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:06.954568 17140 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:06.937199 17106 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.849s	user 0.013s	sys 0.009s
W20260709 13:03:06.937811 17167 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.893s	user 0.005s	sys 0.016s
I20260709 13:03:06.944025 15086 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.453685 15085 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:06.946516 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.947676 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.989080 15160 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:06.963742 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:06.952790 17168 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.008s
I20260709 13:03:06.970409 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.970748 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.994933 15084 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:06.971557 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.945322 17117 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:06.954245 17008 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.543s	user 0.019s	sys 0.023s
I20260709 13:03:06.972643 15083 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:06.975543 15081 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:06.977218 15128 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.730s (SENT)
I20260709 13:03:07.010612 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:06.949131 17115 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.978194 15088 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:06.953246 17030 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:07.013183 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.014025 15084 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:06.979079 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2300) took 1056 ms. Trace:
I20260709 13:03:06.524592 17110 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:06.982343 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46301.2us
I20260709 13:03:06.982959 15162 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:06.991964 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:07.007194 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64646.3us
I20260709 13:03:07.009104 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:06.977818 17139 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:06.955703 17170 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.972s	user 0.011s	sys 0.017s
I20260709 13:03:06.983287 17113 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4.
I20260709 13:03:06.981751 15079 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:07.022521 15161 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:07.008126 17168 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:06.980113 17187 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.984025 17140 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.015767 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.017374 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:06.979861 15843 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:07.032966 15082 raft_consensus.cc:2368] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3.
I20260709 13:03:07.016749 15841 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:07.013137 17115 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.035878 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.035419 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:06.994637 17106 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.018756 17035 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.023608 17152 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.019150 17110 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:07.015323 17008 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.037354 15237 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:07.118897 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.122553 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.191402 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.367208 15230 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:07.225019 15086 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
W20260709 13:03:07.097234 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 116191us
W20260709 13:03:07.191785 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 160493us
I20260709 13:03:07.019799 17198 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:07.262778 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:07.388108 15232 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:07.198433 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 124822us
I20260709 13:03:07.374357 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:07.190321 14983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 365) took 4418 ms. Trace:
W20260709 13:03:07.218014 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 254329us
I20260709 13:03:07.038040 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.384511 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:07.086622 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.386420 17204 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:07.386518 17212 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:07.393016 15158 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.392542 15229 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:07.104364 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.234104 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.035387 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.210279 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:07.454921 15844 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:07.450346 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:07.329761 16998 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.016s
I20260709 13:03:07.454560 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:07.034140 17187 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.458597 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1558) took 1043 ms. Trace:
W20260709 13:03:07.458578 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64221.2us
I20260709 13:03:07.362609 17019 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.381047 17110 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:07.357889 17154 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.906s	user 0.013s	sys 0.003s
I20260709 13:03:07.388338 17198 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.493973 15848 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:07.029515 17170 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:07.392344 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 432686us
W20260709 13:03:07.459896 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67671.7us
W20260709 13:03:07.321594 17158 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.971s	user 0.012s	sys 0.008s
I20260709 13:03:07.034402 17140 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.300021 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:07.510015 17009 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:07.191313 17142 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:07.514617 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56750.9us
I20260709 13:03:07.380884 17177 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.034305 17181 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:07.340814 17169 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.253s	user 0.010s	sys 0.022s
I20260709 13:03:07.387780 17097 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.509572 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:07.501605 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.518483 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.237259 17188 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:07.365844 17114 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.971s	user 0.020s	sys 0.009s
W20260709 13:03:07.391192 15201 connection.cc:256] RPC call timeout handler was delayed by -1.0862065770000982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:03:07.647859 15201 connection.cc:256] RPC call timeout handler was delayed by -1.0842964200001006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 13:03:07.221755 17190 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:03:07.651779 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2156) took 2008 ms. Trace:
I20260709 13:03:07.652175 15083 rpcz_store.cc:276] 0709 13:03:05.643420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.080951 (+437531us) service_pool.cc:224] Handling call
0709 13:03:07.651584 (+1570633us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.651716 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972216704}
I20260709 13:03:07.652688 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2168) took 1362 ms. Trace:
I20260709 13:03:07.653375 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:07.654515 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:07.655529 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:07.656219 15083 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:07.656116 17114 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.657815 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.653088 15087 rpcz_store.cc:276] 0709 13:03:06.292880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.314767 (+ 21887us) service_pool.cc:224] Handling call
0709 13:03:06.333961 (+ 19194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.339247 (+  5286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.339296 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.404870 (+ 65574us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.509384 (+104514us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:06.509442 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.509549 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.652484 (+1142935us) raft_consensus.cc:1685] finished
0709 13:03:07.652502 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.652617 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11051,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":103965,"threads_started":1,"wal-append.queue_time_us":135785}
I20260709 13:03:07.659597 15087 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:03:07.554061 17030 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.001s
I20260709 13:03:07.581626 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 710) took 1047 ms. Trace:
I20260709 13:03:07.580904 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 678) took 1025 ms. Trace:
I20260709 13:03:07.545624 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:07.529597 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 563346us
I20260709 13:03:07.586886 15232 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
W20260709 13:03:07.504705 17053 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.156s	user 0.008s	sys 0.021s
I20260709 13:03:07.033859 17139 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.513588 15088 rpcz_store.cc:276] 0709 13:03:06.415111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.563391 (+148280us) service_pool.cc:224] Handling call
0709 13:03:07.458400 (+895009us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b540011a3d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.458535 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64993024}
I20260709 13:03:07.382916 17200 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.450110 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 665) took 1320 ms. Trace:
I20260709 13:03:07.550031 15228 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:07.394521 17168 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:07.349931 17189 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.312s	user 0.000s	sys 0.000s
I20260709 13:03:07.440367 17093 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.548058 17154 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.577275 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 664) took 1448 ms. Trace:
I20260709 13:03:07.554140 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:07.215956 17117 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:07.451720 17008 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.643528 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:07.644587 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51728.2us
I20260709 13:03:07.588301 15234 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:07.592483 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.649127 15230 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:07.649610 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:07.650485 15126 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.680s (SENT)
I20260709 13:03:07.643604 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.660820 15084 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:07.663244 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 117722us
I20260709 13:03:07.667418 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.668426 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:07.664081 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 692) took 2228 ms. Trace:
I20260709 13:03:07.661487 15843 rpcz_store.cc:276] 0709 13:03:06.534370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.205029 (+670659us) service_pool.cc:224] Handling call
0709 13:03:07.581550 (+376521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.661810 15840 rpcz_store.cc:276] 0709 13:03:06.531029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.014733 (+483704us) service_pool.cc:224] Handling call
0709 13:03:07.556341 (+541608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.661157 17030 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.673257 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:07.674577 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.547950 17198 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:07.512511 16998 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:07.579775 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 373) took 1879 ms (client timeout 2117 ms). Trace:
I20260709 13:03:07.647357 15847 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:07.591688 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.581741 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:07.725651 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.726145 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.726862 15845 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.728948 15841 rpcz_store.cc:276] 0709 13:03:05.435582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.439015 (+  3433us) service_pool.cc:224] Handling call
0709 13:03:06.349568 (+910553us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:06.349738 (+   170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.086595 (+736857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.663830 (+577235us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540074e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:07.664005 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464032512}
I20260709 13:03:07.730432 16998 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.731154 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.731659 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.732342 15841 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:07.732496 15157 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.733871 15161 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:07.734850 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.735371 15161 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:07.736523 15160 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:07.737486 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:07.739460 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89938.4us
I20260709 13:03:07.578500 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:07.680204 15158 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
W20260709 13:03:07.647531 17132 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.001s
I20260709 13:03:07.680998 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.744377 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 716) took 1116 ms. Trace:
I20260709 13:03:07.681819 15237 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.648912 17169 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.552119 17009 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:07.744721 15846 rpcz_store.cc:276] 0709 13:03:06.628155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.665342 (+1037187us) service_pool.cc:224] Handling call
0709 13:03:07.744320 (+ 78978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:07.121834 14985 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.693356 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 667) took 1562 ms. Trace:
I20260709 13:03:07.645900 17188 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:07.550622 17158 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.661904 17114 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:07.750785 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 82440.2us
I20260709 13:03:07.663141 17202 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:07.751668 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 103364us
I20260709 13:03:07.662675 17210 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:07.699082 15155 raft_consensus.cc:2368] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:07.679282 17204 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:07.738761 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:07.740325 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:07.733228 15162 rpcz_store.cc:269] 0709 13:03:05.700597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.869954 (+169357us) service_pool.cc:224] Handling call
0709 13:03:07.579574 (+1709620us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002878d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.579713 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556768512}
W20260709 13:03:07.687287 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 232949us
W20260709 13:03:07.789870 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49590.4us
I20260709 13:03:07.790125 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:07.790500 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:07.744715 15160 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:03:07.745491 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 167073us
I20260709 13:03:07.746013 15158 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:07.613864 17181 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:07.746685 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 96343.4us
I20260709 13:03:07.794067 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:07.746984 15080 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:07.785334 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.748015 15844 rpcz_store.cc:276] 0709 13:03:06.130712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.130821 (+   109us) service_pool.cc:224] Handling call
0709 13:03:07.692765 (+1561944us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540066f8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.693297 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718078336}
I20260709 13:03:07.391784 14983 rpcz_store.cc:276] 0709 13:03:02.524611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.524773 (+   162us) service_pool.cc:224] Handling call
0709 13:03:06.637603 (+4112830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:04.937337 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:04.970427 (+ 33090us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:04.970484 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:03:04.970526 (+    42us) tablet.cc:662] Decoding operations
0709 13:03:04.970988 (+   462us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:04.971087 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:04.971141 (+    54us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:03:04.971496 (+   355us) tablet.cc:701] Row locks acquired
0709 13:03:04.971550 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:03:04.971792 (+   242us) write_op.cc:270] Start()
0709 13:03:04.971961 (+   169us) write_op.cc:276] Timestamp: P: 1783602184971702 usec, L: 0
0709 13:03:04.971989 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:03:05.047956 (+ 75967us) log.cc:844] Serialized 539 byte log entry
0709 13:03:05.873169 (+825213us) op_driver.cc:464] REPLICATION: finished
0709 13:03:06.085323 (+212154us) write_op.cc:301] APPLY: starting
0709 13:03:06.085428 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:06.085564 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:06.085703 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:06.115118 (+ 29415us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:06.156746 (+ 41628us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:03:06.156803 (+    57us) write_op.cc:312] APPLY: finished
0709 13:03:06.391991 (+235188us) log.cc:844] Serialized 23 byte log entry
0709 13:03:06.429012 (+ 37021us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:06.429093 (+    81us) write_op.cc:454] Released schema lock
0709 13:03:06.429150 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:06.429573 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:06.429615 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":170777,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":193429,"apply.run_cpu_time_us":2969,"apply.run_wall_time_us":344219,"num_ops":1,"prepare.queue_time_us":593472,"prepare.run_cpu_time_us":28162,"prepare.run_wall_time_us":402091,"raft.queue_time_us":314894,"raft.run_cpu_time_us":24371,"raft.run_wall_time_us":198498,"replication_time_us":869323,"thread_start_us":609918,"threads_started":4,"wal-append.queue_time_us":353748,"wal-append.run_cpu_time_us":52346,"wal-append.run_wall_time_us":2228430}]]}
I20260709 13:03:07.570493 17106 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:07.682399 17053 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.556206 17212 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.815332 17158 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.815867 17212 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.742630 16946 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:07.681567 15839 rpcz_store.cc:276] 0709 13:03:06.130156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.130302 (+   146us) service_pool.cc:224] Handling call
0709 13:03:07.387699 (+1257397us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540049c0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.450002 (+ 62303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274873216}
I20260709 13:03:07.693368 17200 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.824970 15154 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.825935 17170 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.817435 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.829366 15839 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:07.829267 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:07.830389 17168 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:07.830535 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.830845 17168 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.831142 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:07.831902 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.832516 15839 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.015453 15156 rpcz_store.cc:276] 0709 13:03:05.922683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.099148 (+176465us) service_pool.cc:224] Handling call
0709 13:03:06.099466 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.099799 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:03:06.324540 (+224741us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.549443 (+224903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.944232 (+394789us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:06.954419 (+ 10187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.960960 (+  6541us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.976527 (+ 15567us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":279356,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":13125,"raft.run_cpu_time_us":11670,"raft.run_wall_time_us":391220,"thread_start_us":354553,"threads_started":2,"wal-append.queue_time_us":658742}
I20260709 13:03:07.833809 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.834009 17206 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:07.834390 15156 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:07.765259 17030 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.835592 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:07.765995 17117 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.837360 17035 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:07.837736 17035 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:07.830555 17209 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:03:07.838395 17035 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.839268 17110 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:07.840553 15847 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:07.773985 17115 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.777037 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.846067 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:07.778697 17139 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.780274 17154 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.785306 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.752315 15085 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:07.787915 15157 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:07.752671 17188 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.794219 15845 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:07.750958 16998 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:07.796082 15161 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:07.750174 17211 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:07.797791 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59080.8us
I20260709 13:03:07.751353 17009 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:07.694849 15842 rpcz_store.cc:276] 0709 13:03:06.129192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.129325 (+   133us) service_pool.cc:224] Handling call
0709 13:03:06.129676 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.537176 (+407500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.577026 (+1039850us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b540041c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:07.577214 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747135744}
I20260709 13:03:07.758499 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:07.750187 17132 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.801270 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.802011 17152 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:07.802461 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1562) took 1386 ms. Trace:
I20260709 13:03:07.803083 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.805032 17208 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:07.804970 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.806841 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.807643 17195 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:07.772038 15840 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:07.809036 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.798539 17177 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:03:07.812134 17181 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.812664 17072 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:07.756553 17202 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.818176 15163 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:07.812999 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:07.817366 17053 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:07.825027 17214 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:07.763759 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.831246 17168 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.833956 17198 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.769042 14985 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.764106 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.841471 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:07.839852 17140 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:07.771552 15237 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:07.846560 17115 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:07.853507 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:07.855242 17187 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.174101 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:07.862511 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64436.2us
W20260709 13:03:08.176163 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 345099us
I20260709 13:03:08.174939 15847 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:08.176550 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1696) took 1203 ms. Trace:
W20260709 13:03:08.176831 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 335397us
I20260709 13:03:08.178757 15846 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:08.179751 15844 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.180398 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1694) took 1223 ms. Trace:
W20260709 13:03:07.919523 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78103.8us
I20260709 13:03:08.180797 15157 rpcz_store.cc:276] 0709 13:03:06.956770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.591429 (+634659us) service_pool.cc:224] Handling call
0709 13:03:08.180343 (+588914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.181857 15160 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.178283 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 656) took 1161 ms. Trace:
W20260709 13:03:08.141849 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 344996us
I20260709 13:03:08.183578 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.184760 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.177796 15161 rpcz_store.cc:276] 0709 13:03:06.973197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.734626 (+761429us) service_pool.cc:224] Handling call
0709 13:03:08.176479 (+441853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.182430 15845 rpcz_store.cc:276] 0709 13:03:07.017319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.725863 (+708544us) service_pool.cc:224] Handling call
0709 13:03:08.178200 (+452337us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:08.177120 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 423324us
W20260709 13:03:08.185715 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 367228us
I20260709 13:03:08.187139 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:08.191335 15163 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 377) took 1911 ms (client timeout 1717 ms). Trace:
I20260709 13:03:08.179970 15080 rpcz_store.cc:276] 0709 13:03:06.416282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.560515 (+144233us) service_pool.cc:224] Handling call
0709 13:03:07.802261 (+1241746us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:07.802403 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1819773184}
W20260709 13:03:08.189159 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 395075us
W20260709 13:03:08.197707 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 403685us
I20260709 13:03:08.215260 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:08.215485 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.847226 17213 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:03:08.213750 15163 rpcz_store.cc:269] 0709 13:03:06.279762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.279930 (+   168us) service_pool.cc:224] Handling call
0709 13:03:08.191162 (+1911232us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003ffcd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:08.191275 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3458402048}
I20260709 13:03:08.212339 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.212698 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.174937 17187 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.188328 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.228238 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:08.228804 15158 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:08.189023 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:08.196791 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.229785 15845 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:08.005034 17180 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.025s	user 0.006s	sys 0.002s
I20260709 13:03:08.220928 17170 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:08.223078 15051 connection.cc:256] RPC call timeout handler was delayed by -1.4578630450000674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 13:03:08.184674 17214 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.227800 15157 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:08.231264 15051 connection.cc:256] RPC call timeout handler was delayed by -1.0339630450000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
I20260709 13:03:08.128284 17110 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.184950 17202 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:08.238986 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50034.1us
I20260709 13:03:08.232674 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.176744 17152 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:08.234272 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.177680 17208 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.235666 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.179104 17195 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:08.188999 15843 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:07.847350 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.259182 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.257457 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.225521 17168 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.187848 17115 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:08.184082 17200 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:08.256275 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 71586.3us
I20260709 13:03:08.191661 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 654) took 1171 ms. Trace:
I20260709 13:03:08.238729 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.234686 17214 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.283860 15080 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.191444 17114 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.233290 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:08.225910 16946 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:08.240893 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.820426 17210 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.250072 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.252239 15083 raft_consensus.cc:2368] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5.
I20260709 13:03:08.226316 17178 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:07.852344 17035 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.256954 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.238067 16998 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:08.310667 15847 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:08.259601 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.236208 17209 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:08.259222 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.316818 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.258436 17195 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.254436 15157 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:08.260991 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.257522 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.269146 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.269786 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.269932 17198 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:08.250631 17211 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:08.308734 17191 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:08.274905 15086 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.275393 17132 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.254714 17169 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.248378 17106 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:08.314417 16998 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:08.236773 17180 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.261831 17115 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.291945 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.284265 15840 rpcz_store.cc:276] 0709 13:03:07.016703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.725208 (+708505us) service_pool.cc:224] Handling call
0709 13:03:08.187656 (+462448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.294438 17097 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:08.340291 17097 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.341598 17097 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.342367 17168 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.292181 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 383) took 1352 ms. Trace:
I20260709 13:03:08.295611 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:08.297183 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83212.8us
I20260709 13:03:08.343918 15159 rpcz_store.cc:276] 0709 13:03:06.939576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.939709 (+   133us) service_pool.cc:224] Handling call
0709 13:03:06.957216 (+ 17507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.125896 (+168680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.125934 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:07.181177 (+ 55243us) log.cc:844] Serialized 26 byte log entry
0709 13:03:07.524481 (+343304us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:07.524539 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.524622 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.291966 (+767344us) raft_consensus.cc:1685] finished
0709 13:03:08.291990 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.292124 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":420056,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":430206,"threads_started":2,"wal-append.queue_time_us":397138}
I20260709 13:03:08.298609 15160 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:08.345818 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.346576 15846 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.307343 15080 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
W20260709 13:03:08.346954 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 118783us
I20260709 13:03:08.347690 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.348032 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:08.247650 17152 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.254401 17187 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.317895 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.228696 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.319931 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.343384 17097 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.330318 17191 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:08.353644 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:08.323693 15082 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.263195 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:08.325614 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90971.9us
I20260709 13:03:08.326568 17222 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:08.273849 17140 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:08.284488 15839 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:08.309419 17178 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:08.330967 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.313987 17200 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.331879 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.252547 17208 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.322055 17114 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:08.338260 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:07.819820 17204 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.348323 15080 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:08.311750 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61746.7us
I20260709 13:03:08.329360 17211 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:08.321287 15845 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:08.345227 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.338608 17115 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:08.322856 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83172us
I20260709 13:03:08.337522 16998 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:08.381191 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 152563us
I20260709 13:03:08.385905 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.354769 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.334053 17106 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.429306 15846 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:08.380072 17191 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.396620 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:08.338383 17198 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:08.394057 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.395951 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:08.394649 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.397274 15841 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:08.395052 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78329.3us
I20260709 13:03:08.506129 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.393292 15088 raft_consensus.cc:2376] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4. Re-sending same reply.
I20260709 13:03:08.488881 17132 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.488440 17188 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.489210 15162 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:08.507823 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 111976us
W20260709 13:03:08.489584 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103800us
I20260709 13:03:08.489753 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.508813 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:08.489464 17106 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.513355 15128 leader_election.cc:400] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5.
I20260709 13:03:08.543797 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:08.544267 15128 leader_election.cc:403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:08.490509 15054 leader_election.cc:400] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:08.544806 15054 leader_election.cc:403] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:08.490686 15084 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:08.491097 15085 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 2: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:08.491474 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 146780us
I20260709 13:03:08.392315 17168 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.492830 17204 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.493227 17211 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.494050 17178 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.495040 17114 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.549767 17035 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.397370 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.484557 15847 raft_consensus.cc:2376] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
I20260709 13:03:08.497427 15846 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.582628 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.586712 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 679) took 2044 ms. Trace:
I20260709 13:03:08.587150 15846 rpcz_store.cc:276] 0709 13:03:06.542304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.496919 (+1954615us) service_pool.cc:224] Handling call
0709 13:03:08.497254 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.586467 (+ 89213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.586661 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.588359 15846 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.499065 15844 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:08.588441 17202 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.588984 16946 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:08.499596 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.544745 17181 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:08.589864 15080 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.589814 17181 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:08.485289 15081 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.540717 17218 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:03:08.387380 15154 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.004s
I20260709 13:03:08.498014 15127 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.590924 17202 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:08.508080 15231 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.426216 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:08.591372 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 108144us
I20260709 13:03:08.499337 17008 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:08.592114 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.592270 17008 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:08.593019 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.490234 17187 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:08.484856 17072 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:08.527238 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.394662 15842 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:08.542572 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 157553us
I20260709 13:03:08.597385 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 683) took 1660 ms. Trace:
I20260709 13:03:08.597815 15846 rpcz_store.cc:276] 0709 13:03:06.934366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.587837 (+1653471us) service_pool.cc:224] Handling call
0709 13:03:08.588194 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.588806 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.594977 (+  6171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.547024 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:08.497427 17210 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.550892 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.554831 15841 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.539872 17110 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:08.561715 16998 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.607277 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.522823 17115 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.560261 15843 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:08.559512 17209 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:08.569945 17180 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:08.549371 17221 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.552815 17132 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5.
I20260709 13:03:08.570276 17009 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.547597 17217 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.570489 15162 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:08.556005 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65639.5us
I20260709 13:03:08.624264 17211 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.410913 17169 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:08.548813 15156 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.001s
I20260709 13:03:08.642436 15842 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:08.642691 15156 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:08.489488 17214 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.643563 15845 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:08.644019 17132 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.644531 17132 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:08.645138 17214 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.506311 17200 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:03:08.645481 17132 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.646301 17200 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:08.646694 17200 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.647557 17200 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.648522 17009 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:08.649216 17217 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:08.649493 17169 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:08.650206 17217 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.650465 17169 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.614282 17139 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.590299 17181 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.590720 17218 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:08.582863 17178 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:08.498875 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 102378us
I20260709 13:03:08.500917 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:08.601588 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54628.1us
I20260709 13:03:08.578331 17114 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:08.609247 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58427.4us
I20260709 13:03:08.610791 15846 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:08.602165 17154 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.613204 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.675529 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 684) took 1740 ms. Trace:
I20260709 13:03:08.602358 17053 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.597051 17204 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.617839 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 682) took 1683 ms. Trace:
I20260709 13:03:08.622294 17188 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.622543 17106 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.623355 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 664) took 1070 ms. Trace:
I20260709 13:03:08.626924 17115 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.628544 16998 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.593273 17008 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.629613 17140 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:08.605715 17216 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.632158 15087 raft_consensus.cc:2368] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:08.679392 17180 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:08.633754 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 73952.7us
I20260709 13:03:08.595765 15154 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:08.638723 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.624614 17208 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.589125 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 660) took 1077 ms. Trace:
I20260709 13:03:08.588354 17152 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.658942 15848 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:08.661576 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55290.7us
I20260709 13:03:08.668463 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:08.668826 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.648767 17200 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.638350 17221 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:08.673352 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 172543us
I20260709 13:03:08.636675 17168 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.675263 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 401) took 1345 ms. Trace:
I20260709 13:03:08.676167 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 680) took 2054 ms. Trace:
I20260709 13:03:08.676285 15846 rpcz_store.cc:276] 0709 13:03:06.934661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.598657 (+1663996us) service_pool.cc:224] Handling call
0709 13:03:08.610633 (+ 11976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.675147 (+ 64514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.675478 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.677656 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.681185 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.705379 15234 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.692401 17140 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:08.685292 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.710939 15230 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.662587 17218 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.687819 15841 rpcz_store.cc:276] 0709 13:03:06.934105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.543084 (+1608979us) service_pool.cc:224] Handling call
0709 13:03:08.549383 (+  6299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.608655 (+ 59272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.617768 (+  9113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.633380 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 659) took 1393 ms. Trace:
I20260709 13:03:08.689771 15843 rpcz_store.cc:276] 0709 13:03:07.552644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.756135 (+203491us) service_pool.cc:224] Handling call
0709 13:03:08.623271 (+867136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:08.694222 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.688551 17053 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:08.697088 15844 rpcz_store.cc:276] 0709 13:03:07.511728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.802804 (+291076us) service_pool.cc:224] Handling call
0709 13:03:08.589067 (+786263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.715065 15847 rpcz_store.cc:276] 0709 13:03:07.240427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.608415 (+1367988us) service_pool.cc:224] Handling call
0709 13:03:08.632721 (+ 24306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.632976 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.633014 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.633069 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.633113 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.633184 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.633309 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.701881 16998 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:03:08.699604 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60970.6us
I20260709 13:03:08.686520 17154 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:08.688709 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.701881 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.705865 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.705963 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.697806 15079 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.704356 15231 rpcz_store.cc:276] 0709 13:03:07.330413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.453227 (+122814us) service_pool.cc:224] Handling call
0709 13:03:07.459830 (+  6603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.591688 (+1131858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.675025 (+ 83337us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000ac8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:08.675199 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410222464}
I20260709 13:03:08.705029 15845 rpcz_store.cc:276] 0709 13:03:06.621625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.510078 (+1888453us) service_pool.cc:224] Handling call
0709 13:03:08.643384 (+133306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.643935 (+   551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.675995 (+ 32060us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b540001a4d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:08.676113 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223227648}
I20260709 13:03:08.682653 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.707755 17168 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.709400 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.740200 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.696466 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.692090 15232 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.702768 17198 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:08.717036 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.744558 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.744952 15844 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.699085 15162 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.718269 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.722000 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.726007 17202 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:08.726653 17225 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:08.732787 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:08.735833 15230 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:08.738415 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.738734 17053 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.708240 17221 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.755785 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:08.755899 17238 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.749567 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.722815 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.765729 15234 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:08.728559 16998 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:08.772337 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63027us
I20260709 13:03:08.740531 17139 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.774675 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.719509 17180 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:03:08.777715 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.778967 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.754392 17198 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.782545 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:08.716665 17224 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:03:08.745121 17216 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:08.784168 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.765233 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.797206 17097 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:08.725191 17231 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:03:08.799196 17180 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:08.799646 17180 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:08.800189 17117 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.800716 17117 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.800447 17180 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.801352 17097 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.801755 17180 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.801664 17117 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.802337 17097 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:08.721868 17191 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.797760 17216 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.808096 15162 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:08.805963 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:08.810475 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 84354.5us
I20260709 13:03:08.811431 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:08.753821 17232 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:08.812990 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.809603 15844 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:08.812742 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.814357 15234 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:08.758388 17230 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:08.822007 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45116.2us
I20260709 13:03:08.820735 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.822741 15080 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.825830 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:08.826825 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.781610 15236 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.830549 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.831173 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.832163 15237 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:08.832933 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.833942 15228 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:08.754026 17210 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:03:08.779953 17214 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:08.829962 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.785518 17187 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.804921 17198 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.838346 15158 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.845044 15084 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:08.838440 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.839874 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.847579 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:08.784727 17204 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:08.765618 17202 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.841735 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.858115 15157 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.850602 15080 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:08.797786 15841 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:08.856495 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.823472 17115 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.869575 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:08.871768 15162 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.873661 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.766398 17195 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:08.827837 17211 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.875818 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.841830 17177 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.842393 17209 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.883683 15237 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:08.889146 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:08.843281 17230 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:08.895036 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:08.857275 17225 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:09.033030 15158 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:09.037632 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.869656 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:09.043412 15126 leader_election.cc:336] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.651s (SENT)
I20260709 13:03:09.056917 15083 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.876686 17214 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:09.065680 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:09.066601 15081 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:09.067210 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:09.073773 17242 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:09.077555 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:08.897405 17110 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.883028 17115 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:09.058246 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:08.896875 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:09.095539 15083 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:03:09.093830 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 225787us
I20260709 13:03:08.908931 17106 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:08.878057 17234 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:09.043018 17225 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:08.804260 17228 consensus_queue.cc:1175] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 13:03:09.139876 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:09.119777 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:09.067167 17035 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:09.054641 17152 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:09.056957 17177 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:09.173127 15157 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:09.099465 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:09.188256 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.190652 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.204165 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:09.196156 15157 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:09.196992 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.208868 15161 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:08.882666 17154 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.108814 17139 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:09.104444 17208 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:09.113225 17187 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:09.025996 16998 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:09.172037 17225 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:09.237110 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:09.243657 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.002s	sys 0.005s
W20260709 13:03:09.253156 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.002s	sys 0.005s
I20260709 13:03:09.143714 15156 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:09.255550 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:09.171959 17035 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:09.098765 17211 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 13:03:09.246387 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:09.246286 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 126602us
I20260709 13:03:09.250252 15157 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:09.250571 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 403079us
I20260709 13:03:09.244992 17191 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:09.105329 17110 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.255613 15162 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:09.233173 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:09.263911 15157 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:09.265259 17232 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:09.299564 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:09.303200 17214 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:09.303015 17232 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:09.303766 17214 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:09.250065 17233 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:09.305744 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.174098 17251 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.143s
W20260709 13:03:09.315979 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.316659 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.317281 15162 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:09.322610 15157 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:09.193507 17204 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.330892 17253 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:03:09.332306 17236 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:03:09.252123 17230 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:09.333603 17114 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:03:09.334030 17114 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:09.307927 17214 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.359114 15162 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:09.308917 17246 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:09.362921 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.328686 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:09.259666 17177 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.336790 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.355209 17202 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:09.354385 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:09.377590 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.306756 17115 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:09.375847 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.014s	sys 0.013s
I20260709 13:03:09.363823 17195 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:09.345152 17187 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:09.350414 17035 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.513834 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.489419 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:09.503942 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:09.521096 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.019s	sys 0.022s
I20260709 13:03:09.522634 15158 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:09.523767 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 790988us
W20260709 13:03:09.529098 15203 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items
I20260709 13:03:09.537221 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:09.464423 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.002s	sys 0.001s
I20260709 13:03:09.373364 17114 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:09.499025 15156 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:09.541558 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.004s	sys 0.006s
I20260709 13:03:09.520715 17231 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:09.545553 17231 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:09.546412 17231 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.547431 17231 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:09.542745 15847 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:09.498132 17188 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:03:09.544689 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 649586us
I20260709 13:03:09.439656 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:09.439580 15236 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.000s
I20260709 13:03:09.375752 17246 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.367899 17139 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.533218 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:09.568264 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:09.914331 15157 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.591s	user 0.017s	sys 0.063s
I20260709 13:03:09.915091 15157 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
W20260709 13:03:09.860037 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.718472 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:09.916590 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:09.917448 15847 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:09.918485 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:09.918311 15128 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:09.919281 15080 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:09.855412 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.920415 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.664156 15081 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 13:03:09.921407 15081 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:09.659098 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51793.6us
I20260709 13:03:09.845166 15844 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:03:09.463599 17200 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.759s	user 0.007s	sys 0.002s
W20260709 13:03:09.928567 15126 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.672s (SENT)
I20260709 13:03:09.928716 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:09.913081 17225 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.002s
I20260709 13:03:09.930068 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 689) took 1067 ms. Trace:
I20260709 13:03:09.935173 15847 rpcz_store.cc:276] 0709 13:03:08.862423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.567928 (+705505us) service_pool.cc:224] Handling call
0709 13:03:09.930001 (+362073us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.689611 17180 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.887s	user 0.008s	sys 0.004s
I20260709 13:03:09.930874 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:09.931557 15086 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
W20260709 13:03:09.680557 17181 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.011s	user 0.010s	sys 0.002s
W20260709 13:03:09.727303 17218 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.006s	user 0.007s	sys 0.003s
W20260709 13:03:09.707340 17249 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:03:09.933363 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:09.549764 17035 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:09.706157 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:09.820959 16946 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:09.577767 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:09.942206 15126 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.159s (SENT)
I20260709 13:03:09.941174 15237 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:09.917161 15236 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:09.617468 17110 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:09.929745 17234 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:03:09.960003 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.963837 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.922140 17188 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:09.974000 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:09.974614 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:09.965518 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:09.976704 15847 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:09.938324 17258 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:09.977794 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 690) took 1115 ms. Trace:
I20260709 13:03:09.982337 15847 rpcz_store.cc:276] 0709 13:03:08.862644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.951206 (+1088562us) service_pool.cc:224] Handling call
0709 13:03:09.977737 (+ 26531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.960913 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1590) took 1087 ms. Trace:
I20260709 13:03:09.946242 17232 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:09.932085 17228 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:09.980278 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:09.954299 17210 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:09.968032 17242 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.893s	user 0.014s	sys 0.000s
I20260709 13:03:09.539044 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1591)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1574)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1584)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1585)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1588)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1589)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
I20260709 13:03:09.967763 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 687) took 1105 ms. Trace:
I20260709 13:03:09.723536 17140 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.939522 17225 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.008023 17225 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:09.988373 17264 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:10.010521 17178 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.341s	user 0.010s	sys 0.002s
W20260709 13:03:10.011199 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 713) took 3475 ms (client timeout 2171 ms). Trace:
I20260709 13:03:10.012049 15840 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:10.011855 15848 rpcz_store.cc:269] 0709 13:03:06.535269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.552063 (+1016794us) service_pool.cc:224] Handling call
0709 13:03:10.010786 (+2458723us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b5400498c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.011015 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156600064}
I20260709 13:03:10.012701 17220 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.013337 17220 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.013756 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:10.014777 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 718) took 1348 ms. Trace:
I20260709 13:03:10.015558 15839 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:10.015218 15840 rpcz_store.cc:276] 0709 13:03:08.666523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.669637 (+  3114us) service_pool.cc:224] Handling call
0709 13:03:10.014541 (+1344904us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54007e4354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.014711 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2919502080}
I20260709 13:03:09.976558 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 410) took 1263 ms. Trace:
I20260709 13:03:10.014911 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 692) took 1151 ms. Trace:
I20260709 13:03:09.950755 17181 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.983767 15233 rpcz_store.cc:276] 0709 13:03:08.866042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.439342 (+573300us) service_pool.cc:224] Handling call
0709 13:03:09.953282 (+513940us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.999715 15085 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:09.991863 15841 rpcz_store.cc:276] 0709 13:03:08.861874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.498439 (+636565us) service_pool.cc:224] Handling call
0709 13:03:09.963613 (+465174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.005690 17198 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.124s	user 0.012s	sys 0.000s
I20260709 13:03:09.951051 17218 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.978494 17234 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.019100 15848 rpcz_store.cc:276] 0709 13:03:08.863078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.013357 (+1150279us) service_pool.cc:224] Handling call
0709 13:03:10.014836 (+  1479us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.970366 16946 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:09.985137 17232 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:09.986470 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 732) took 1273 ms. Trace:
I20260709 13:03:10.080693 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:09.992192 17242 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.017185 15236 rpcz_store.cc:276] 0709 13:03:08.706738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.710093 (+  3355us) service_pool.cc:224] Handling call
0709 13:03:09.970217 (+1260124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.091130 15126 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.574s (SENT)
I20260709 13:03:09.986104 17258 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.221992 15845 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:10.265466 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 285235us
W20260709 13:03:10.232457 15201 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.650s (SENT)
I20260709 13:03:09.979291 17260 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.332832 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:10.182919 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.707428 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1715) took 1171 ms. Trace:
I20260709 13:03:10.817770 15157 rpcz_store.cc:276] 0709 13:03:09.535525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.706334 (+1170809us) service_pool.cc:224] Handling call
0709 13:03:10.706721 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.706967 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.707008 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.707070 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:10.707118 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.707197 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.707352 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.705626 15155 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:10.519687 17181 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:10.360057 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1584)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1585)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1588)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1589)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
I20260709 13:03:10.840750 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2315) took 1464 ms. Trace:
W20260709 13:03:10.524798 17264 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.000s
I20260709 13:03:10.841257 15155 rpcz_store.cc:276] 0709 13:03:09.376922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.704769 (+1327847us) service_pool.cc:224] Handling call
0709 13:03:10.705278 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.840467 (+135189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.840694 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.841557 17264 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.517974 17218 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:10.498008 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.844656 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.395072 15847 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 691) took 1532 ms (client timeout 1931 ms). Trace:
W20260709 13:03:10.537317 17210 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
W20260709 13:03:10.846731 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1588) took 2589 ms (client timeout 1526 ms). Trace:
W20260709 13:03:10.849776 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 759695us
W20260709 13:03:10.502799 17267 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:03:10.840296 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 607835us
W20260709 13:03:10.407232 17266 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 13:03:10.575529 17231 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.981s	user 0.006s	sys 0.002s
I20260709 13:03:10.081938 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:10.847249 15847 rpcz_store.cc:269] 0709 13:03:08.862867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.994699 (+1131832us) service_pool.cc:224] Handling call
0709 13:03:10.394968 (+400269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.849550 15087 rpcz_store.cc:269] 0709 13:03:08.244034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.295291 (+ 51257us) service_pool.cc:224] Handling call
0709 13:03:10.833192 (+2537901us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54001b17d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.833375 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662134016}
W20260709 13:03:10.850162 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.850603 15126 leader_election.cc:336] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.589s (SENT)
W20260709 13:03:10.886701 15126 leader_election.cc:336] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.606s (SENT)
I20260709 13:03:10.396620 17139 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:10.887171 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:10.761632 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1592) took 1889 ms (client timeout 1606 ms). Trace:
I20260709 13:03:10.846324 15128 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:10.324573 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:10.217374 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 565) took 1397 ms. Trace:
I20260709 13:03:10.324467 17200 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:10.888688 17187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.031s	sys 0.028s
I20260709 13:03:10.331241 15843 rpcz_store.cc:276] 0709 13:03:08.712946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.724472 (+ 11526us) service_pool.cc:224] Handling call
0709 13:03:09.705986 (+981514us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:09.983695 (+277709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9854848}
I20260709 13:03:10.890514 17200 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.109185 17198 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.853472 17278 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:10.842345 15203 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
W20260709 13:03:10.894853 15154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1697) took 3916 ms (client timeout 2083 ms). Trace:
W20260709 13:03:10.896288 15154 rpcz_store.cc:269] 0709 13:03:06.980754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.657578 (+676824us) service_pool.cc:224] Handling call
0709 13:03:10.894656 (+3237078us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540037e954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.894771 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369721600}
I20260709 13:03:10.842860 17224 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:10.897157 17224 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:10.890535 17187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.032s	sys 0.028s
I20260709 13:03:10.898041 17224 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.898622 17187 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:10.899559 15846 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:10.900156 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 725) took 2227 ms. Trace:
I20260709 13:03:10.900406 15846 rpcz_store.cc:276] 0709 13:03:08.672523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.896366 (+2223843us) service_pool.cc:224] Handling call
0709 13:03:10.896702 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.899962 (+  3260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.900111 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.408409 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:10.848016 17117 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:10.899015 17208 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:10.243064 17216 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.875958 17008 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:10.295323 17188 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.293789 17228 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:10.294291 17274 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:10.849143 17273 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.414757 17246 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:10.855208 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.888411 15126 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.760s (SENT)
I20260709 13:03:10.384584 17232 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:10.538681 17220 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:10.889312 17260 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:03:10.888782 15236 rpcz_store.cc:269] 0709 13:03:08.866570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.653629 (+1787059us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.756285 (+102656us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.922590 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1579) took 2059 ms (client timeout 1651 ms). Trace:
W20260709 13:03:10.923002 15236 rpcz_store.cc:269] 0709 13:03:08.862872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.922336 (+2059464us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.922498 (+   162us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.923951 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1585) took 2059 ms (client timeout 1650 ms). Trace:
W20260709 13:03:10.924314 15236 rpcz_store.cc:269] 0709 13:03:08.864711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.923748 (+2059037us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.923889 (+   141us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.925323 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 681) took 3991 ms. Trace:
I20260709 13:03:10.925799 15839 rpcz_store.cc:276] 0709 13:03:06.933824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.546495 (+1612671us) service_pool.cc:224] Handling call
0709 13:03:10.015292 (+1468797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.016304 (+  1012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.925122 (+908818us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400498c50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.925260 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3344258816}
I20260709 13:03:10.917938 16946 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:10.927042 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.721s (SENT) [suppressed 90 similar messages]
I20260709 13:03:10.927349 15839 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:10.927132 16946 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:10.928179 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 729) took 2238 ms. Trace:
I20260709 13:03:10.928324 16946 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:10.928771 16946 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:10.929742 16946 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.930286 15200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 420) took 1221 ms. Trace:
I20260709 13:03:10.889603 15085 rpcz_store.cc:276] 0709 13:03:08.819484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.821458 (+  1974us) service_pool.cc:224] Handling call
0709 13:03:09.999204 (+1177746us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfe9c 00007f0d3c8bfa5b 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:09.999428 (+   224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.089033 (+ 89605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.213145 (+124112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40118784}
I20260709 13:03:10.909729 17117 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.936345 17198 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:10.937191 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1566) took 2084 ms (client timeout 1680 ms). Trace:
W20260709 13:03:10.937718 15235 rpcz_store.cc:269] 0709 13:03:08.852815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.936945 (+2084130us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.937097 (+   152us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.928550 15839 rpcz_store.cc:276] 0709 13:03:08.689509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.926805 (+2237296us) service_pool.cc:224] Handling call
0709 13:03:10.927157 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.927875 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.928099 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.938526 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1583) took 2074 ms (client timeout 1672 ms). Trace:
W20260709 13:03:10.938910 15235 rpcz_store.cc:269] 0709 13:03:08.863723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.938369 (+2074646us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.938474 (+   105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.939217 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:10.939977 15154 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:10.912281 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1574) took 2055 ms (client timeout 1606 ms). Trace:
W20260709 13:03:10.941174 15233 rpcz_store.cc:269] 0709 13:03:08.855183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.081907 (+1226724us) service_pool.cc:224] Handling call
0709 13:03:10.910934 (+829027us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.942179 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1587) took 2076 ms (client timeout 1760 ms). Trace:
I20260709 13:03:10.886076 17210 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.939991 15839 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:10.945724 17117 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:10.946197 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 730) took 2250 ms. Trace:
I20260709 13:03:10.946923 15839 rpcz_store.cc:276] 0709 13:03:08.695803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.939013 (+2243210us) service_pool.cc:224] Handling call
0709 13:03:10.939331 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.945685 (+  6354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.946135 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.939764 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1567) took 2086 ms (client timeout 1760 ms). Trace:
W20260709 13:03:10.949450 15235 rpcz_store.cc:269] 0709 13:03:08.853184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.939603 (+2086419us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.939710 (+   107us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.950248 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1588) took 2084 ms (client timeout 1777 ms). Trace:
W20260709 13:03:10.950570 15235 rpcz_store.cc:269] 0709 13:03:08.865543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.950084 (+2084541us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.950193 (+   109us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.902051 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55804.2us
I20260709 13:03:10.895280 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1585)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1588)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1589)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
I20260709 13:03:10.905355 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:10.910240 15846 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:10.899123 17224 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:10.956863 14985 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 319) took 8949 ms. Trace:
W20260709 13:03:10.958257 15086 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2187) took 1917 ms (client timeout 1918 ms). Trace:
I20260709 13:03:10.915324 15841 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:10.908025 17234 raft_consensus.cc:2764] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:03:10.906708 17139 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:10.958689 15086 rpcz_store.cc:269] 0709 13:03:09.040503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.234181 (+193678us) service_pool.cc:224] Handling call
0709 13:03:10.958111 (+1723930us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b540034a154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:10.958204 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553059200}
I20260709 13:03:10.917399 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:10.908633 17208 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:10.899166 17187 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:10.908327 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 693) took 2045 ms (client timeout 2229 ms). Trace:
I20260709 13:03:10.913654 17216 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.919229 17231 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.894726 17200 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:10.954885 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:10.930891 16946 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:10.956528 17035 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.995s	user 0.010s	sys 0.000s
I20260709 13:03:10.956935 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 726) took 2275 ms. Trace:
W20260709 13:03:10.960109 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1580) took 2097 ms (client timeout 1813 ms). Trace:
I20260709 13:03:10.930722 15200 rpcz_store.cc:276] 0709 13:03:09.709125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.930129 (+1221004us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.974116 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 85718us
W20260709 13:03:10.974745 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1570) took 2120 ms (client timeout 1933 ms). Trace:
W20260709 13:03:10.982267 15234 rpcz_store.cc:269] 0709 13:03:08.853987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.974567 (+2120580us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.974678 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:10.987489 15126 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.777s (SENT)
I20260709 13:03:10.976399 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714)
I20260709 13:03:10.988126 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:10.972553 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.979774 15846 rpcz_store.cc:276] 0709 13:03:08.678937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.907929 (+2228992us) service_pool.cc:224] Handling call
0709 13:03:10.910073 (+  2144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.954478 (+ 44405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.954658 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.975375 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1584) took 2111 ms (client timeout 1610 ms). Trace:
W20260709 13:03:10.993530 15231 rpcz_store.cc:269] 0709 13:03:08.863955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.975195 (+2111240us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.975299 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.979233 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:10.978041 17231 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:10.995195 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:10.995846 17238 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:10.996276 17238 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:10.997157 17238 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:10.998144 17238 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:10.998703 17247 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.675s	user 0.013s	sys 0.016s
I20260709 13:03:10.972764 15842 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:10.973888 15154 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:10.974242 17234 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:11.002233 17231 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:10.976797 17187 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:11.003172 15126 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.797s (SENT)
I20260709 13:03:11.003744 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:11.004707 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2308) took 1768 ms (client timeout 1906 ms). Trace:
W20260709 13:03:10.992467 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 62270.5us
W20260709 13:03:11.004802 15126 leader_election.cc:336] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.201s (SENT)
I20260709 13:03:11.003229 17260 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:10.977007 15840 rpcz_store.cc:269] 0709 13:03:08.863287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.017463 (+1154176us) service_pool.cc:224] Handling call
0709 13:03:10.907445 (+889982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.995862 15157 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:11.007175 15126 leader_election.cc:336] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.925s (SENT)
I20260709 13:03:10.974421 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 696) took 1018 ms. Trace:
I20260709 13:03:10.917320 17228 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.014096 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:11.025372 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 98342.4us
W20260709 13:03:11.081163 15051 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.738s (SENT)
W20260709 13:03:10.980144 15235 rpcz_store.cc:269] 0709 13:03:08.863077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.959952 (+2096875us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.960054 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.081707 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.081535 15159 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:11.076064 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:11.005589 15161 rpcz_store.cc:269] 0709 13:03:09.236537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.236728 (+   191us) service_pool.cc:224] Handling call
0709 13:03:11.004477 (+1767749us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540033e854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:11.004640 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547265920}
I20260709 13:03:11.078680 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1718)
I20260709 13:03:11.017959 15847 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.036089 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:11.015446 17241 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.694s	user 0.003s	sys 0.028s
W20260709 13:03:11.085714 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596) took 2218 ms (client timeout 1933 ms). Trace:
I20260709 13:03:11.103916 15163 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:11.022693 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43541.3us
I20260709 13:03:11.151285 15128 leader_election.cc:400] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:11.151687 15128 leader_election.cc:403] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:11.085284 17230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.745s	user 0.020s	sys 0.029s
I20260709 13:03:11.123260 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.123814 15085 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:11.076395 17285 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
W20260709 13:03:11.010342 16998 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.678s	user 0.021s	sys 0.028s
W20260709 13:03:11.166474 16998 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.834s	user 0.025s	sys 0.028s
I20260709 13:03:11.125166 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:11.006222 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1589) took 2126 ms (client timeout 1797 ms). Trace:
I20260709 13:03:11.166963 17220 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.167081 15236 rpcz_store.cc:269] 0709 13:03:08.865792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.965798 (+2100006us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.991824 (+ 26026us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.123312 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:10.973500 17139 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.127684 17187 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:10.974907 17208 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.170621 15080 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.079187 15841 rpcz_store.cc:276] 0709 13:03:09.956014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.214114 (+258100us) service_pool.cc:224] Handling call
0709 13:03:10.974351 (+760237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.118773 17283 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.178952 17283 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:11.130527 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49394.2us
I20260709 13:03:11.181998 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1716) took 1190 ms. Trace:
I20260709 13:03:11.182447 15156 rpcz_store.cc:276] 0709 13:03:09.991674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.180952 (+1189278us) service_pool.cc:224] Handling call
0709 13:03:11.181302 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.181635 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.181668 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.181716 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:11.181751 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.181814 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.181940 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:11.169852 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597) took 2300 ms (client timeout 1934 ms). Trace:
W20260709 13:03:11.184352 15236 rpcz_store.cc:269] 0709 13:03:08.867943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.167935 (+2299992us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.168055 (+   120us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.185331 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599) took 2316 ms (client timeout 1954 ms). Trace:
I20260709 13:03:11.185624 15841 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:11.185736 15236 rpcz_store.cc:269] 0709 13:03:08.879996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.185146 (+2305150us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.185269 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.186292 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 697) took 1229 ms. Trace:
I20260709 13:03:11.186444 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 701) took 1227 ms. Trace:
W20260709 13:03:11.186842 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586) took 2321 ms (client timeout 1959 ms). Trace:
I20260709 13:03:11.006222 17188 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.187709 16946 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:11.187033 15841 rpcz_store.cc:276] 0709 13:03:09.959014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.185067 (+1226053us) service_pool.cc:224] Handling call
0709 13:03:11.185428 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.186183 (+   755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.186389 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.188421 16946 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.189136 15841 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:11.019287 17216 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:10.983783 17274 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:10.944406 15233 rpcz_store.cc:269] 0709 13:03:08.865307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.941996 (+2076689us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:10.942120 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.190047 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 702) took 1230 ms. Trace:
W20260709 13:03:11.190338 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564) took 2354 ms (client timeout 2012 ms). Trace:
I20260709 13:03:11.190769 16946 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:10.918573 17273 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.138743 17228 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.139453 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.145072 15161 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:11.143870 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:11.150534 15235 rpcz_store.cc:269] 0709 13:03:08.867670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.085542 (+2217872us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.085655 (+   113us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.120694 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.197503 15157 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:11.154690 17277 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.200579 17200 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.201067 17277 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.201705 17035 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.204489 17035 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.206686 17035 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.207410 17204 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
W20260709 13:03:11.207924 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568) took 2354 ms (client timeout 1934 ms). Trace:
W20260709 13:03:11.208354 15234 rpcz_store.cc:269] 0709 13:03:08.853475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.207697 (+2354222us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.207843 (+   146us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.155709 17246 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:03:11.157267 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:11.136065 17232 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:11.195428 17218 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.217398 17218 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.218524 17232 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:11.219079 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.219201 15162 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.219504 17218 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:11.187389 15236 rpcz_store.cc:269] 0709 13:03:08.865066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.186640 (+2321574us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.186784 (+   144us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.959682 14985 rpcz_store.cc:276] 0709 13:03:02.007824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.014474 (+  6650us) service_pool.cc:224] Handling call
0709 13:03:10.956756 (+8942282us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:08.425808 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:08.436680 (+ 10872us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:08.436713 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:03:08.436731 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:08.437163 (+   432us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:08.437245 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:08.437277 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:08.458523 (+ 21246us) tablet.cc:701] Row locks acquired
0709 13:03:08.458556 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:03:08.458797 (+   241us) write_op.cc:270] Start()
0709 13:03:08.663237 (+204440us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b5400367fa8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d45da09e1 00007f0d45d9f64c 00007f0d4342f624 00007f0d4341a286 00007f0d43419d58 00007f0d4341e9f1 00007f0d4341e9a9 00007f0d4341e939 00007f0d4341e901 00007f0d4341db2d 000000000038f354
0709 13:03:08.688298 (+ 25061us) write_op.cc:276] Timestamp: P: 1783602188458714 usec, L: 0
0709 13:03:08.690407 (+  2109us) op_driver.cc:348] REPLICATION: starting
0709 13:03:08.719091 (+ 28684us) log.cc:844] Serialized 935 byte log entry
0709 13:03:10.541920 (+1822829us) op_driver.cc:464] REPLICATION: finished
0709 13:03:10.826552 (+284632us) write_op.cc:301] APPLY: starting
0709 13:03:10.826673 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:10.826846 (+   173us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:10.826878 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:10.827477 (+   599us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:10.827739 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:03:10.827809 (+    70us) write_op.cc:312] APPLY: finished
0709 13:03:10.828124 (+   315us) log.cc:844] Serialized 29 byte log entry
0709 13:03:10.829303 (+  1179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:10.829417 (+   114us) write_op.cc:454] Released schema lock
0709 13:03:10.829474 (+    57us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":672494,"reported_tablets":3,"spinlock_wait_cycles":16768,"child_traces":[["op",{"apply.queue_time_us":284200,"num_ops":2,"prepare.queue_time_us":165492,"prepare.run_cpu_time_us":63326,"prepare.run_wall_time_us":406903,"raft.queue_time_us":1086459,"raft.run_cpu_time_us":7287,"raft.run_wall_time_us":344022,"replication_time_us":1851281,"spinlock_wait_cycles":95266688,"thread_start_us":817235,"threads_started":4,"wal-append.queue_time_us":188759}]]}
W20260709 13:03:11.221333 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563) took 2394 ms (client timeout 2041 ms). Trace:
I20260709 13:03:11.109288 17008 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:11.166350 17233 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:11.118793 15839 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:11.222486 17233 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:11.167631 17220 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.222942 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 733) took 2491 ms. Trace:
W20260709 13:03:11.076551 17115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.074s	user 0.014s	sys 0.012s
W20260709 13:03:11.223819 17115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.015s	sys 0.014s
I20260709 13:03:11.223591 17233 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.167702 17140 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.224591 15083 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.224821 17220 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:11.216781 17273 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.125275 15203 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1718)
I20260709 13:03:11.017573 17258 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.177956 17208 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.225839 17035 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:11.241127 17008 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:11.243741 17008 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.203428 17277 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.005030 17278 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:11.190456 15841 rpcz_store.cc:276] 0709 13:03:09.959638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.188551 (+1228913us) service_pool.cc:224] Handling call
0709 13:03:11.188933 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.189764 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.189990 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.249110 17035 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:11.138528 17272 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.133s	user 0.007s	sys 0.004s
I20260709 13:03:11.253465 17035 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:11.149801 17260 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:11.186980 15847 rpcz_store.cc:276] 0709 13:03:09.956730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.954261 (+997531us) service_pool.cc:224] Handling call
0709 13:03:10.974722 (+ 20461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.084562 (+109840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.185964 (+101402us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:11.186219 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99006848}
W20260709 13:03:11.160853 17230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.825s	user 0.025s	sys 0.039s
W20260709 13:03:11.199512 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578) took 2336 ms (client timeout 1996 ms). Trace:
W20260709 13:03:11.259452 15235 rpcz_store.cc:269] 0709 13:03:08.862661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.199328 (+2336667us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.199443 (+   115us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.259771 15088 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:11.209211 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598) took 2341 ms (client timeout 1994 ms). Trace:
I20260709 13:03:11.223371 15839 rpcz_store.cc:276] 0709 13:03:08.732025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.948492 (+2216467us) service_pool.cc:224] Handling call
0709 13:03:11.073300 (+124808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.222647 (+149347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.222867 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:11.260416 15234 rpcz_store.cc:269] 0709 13:03:08.868245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.209041 (+2340796us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.209149 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.195811 15084 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:11.210285 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 66486us
I20260709 13:03:11.262342 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:11.262679 17269 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.239s	user 0.000s	sys 0.011s
I20260709 13:03:11.264103 17258 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:11.211207 15051 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.602s (SENT)
I20260709 13:03:11.265048 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:11.190815 15233 rpcz_store.cc:269] 0709 13:03:08.840967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.190178 (+2349211us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.190283 (+   105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.221714 15236 rpcz_store.cc:269] 0709 13:03:08.835740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.221164 (+2385424us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.221272 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.079144 17234 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:11.264571 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:11.168735 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124904us
W20260709 13:03:11.268217 15201 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.080s (SENT)
I20260709 13:03:11.268740 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:11.201133 17200 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.278121 17230 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:11.278856 17283 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:11.280207 17200 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.280997 17228 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:11.253122 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.258668 15847 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:11.282459 15841 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:11.203255 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 700) took 1244 ms. Trace:
I20260709 13:03:11.283353 15840 rpcz_store.cc:276] 0709 13:03:09.958439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.171003 (+1212564us) service_pool.cc:224] Handling call
0709 13:03:11.202719 (+ 31716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.202905 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.202941 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.202989 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.203039 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.203099 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.203200 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.241221 17233 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.290756 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.292119 17234 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:11.270308 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.293030 17234 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.293671 17234 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.293871 17115 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:11.294567 17115 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.295073 17115 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.293671 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.301620 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.304993 17278 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.305397 17178 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:11.305573 17278 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:11.306537 17178 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:11.306985 17187 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:11.307417 17178 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:11.307482 17187 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.307809 17278 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:11.308053 16946 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:11.308168 17187 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.308452 16946 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:11.308462 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595) took 2441 ms (client timeout 1932 ms). Trace:
W20260709 13:03:11.309955 15231 rpcz_store.cc:269] 0709 13:03:08.867394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.308261 (+2440867us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.308395 (+   134us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.276985 17246 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:11.277192 15201 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.229s (SENT)
I20260709 13:03:11.277640 17273 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:11.292032 17260 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:11.301851 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1591) took 2435 ms (client timeout 1589 ms). Trace:
I20260709 13:03:11.302256 17258 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:11.310690 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559) took 2498 ms (client timeout 2242 ms). Trace:
I20260709 13:03:11.255545 15203 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43960 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1728)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1729)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1724)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1725)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1726)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1727)
I20260709 13:03:11.311364 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:11.326959 15231 rpcz_store.cc:269] 0709 13:03:08.812589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.310539 (+2497950us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.310637 (+    98us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.327353 15157 raft_consensus.cc:2376] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4. Re-sending same reply.
I20260709 13:03:11.328297 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.328914 15157 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:11.312091 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.307819 14984 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.314656 15155 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:11.269891 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58752.7us
I20260709 13:03:11.316098 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:11.330842 15051 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.073s (SENT)
I20260709 13:03:11.331482 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.308403 17278 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.329667 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576) took 2468 ms (client timeout 2200 ms). Trace:
I20260709 13:03:11.306301 17283 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:11.332391 15231 rpcz_store.cc:269] 0709 13:03:08.861286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.329506 (+2468220us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.329614 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.330713 14984 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.337630 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.338500 14984 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.319571 17178 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:11.334600 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.326880 17200 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.344990 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.345271 17200 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:11.323854 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46695.8us
W20260709 13:03:11.324831 15232 rpcz_store.cc:269] 0709 13:03:08.866303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.301661 (+2435358us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.301785 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.338964 15086 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:11.334249 15160 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.340039 15081 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:03:11.334865 15052 connection.cc:256] RPC call timeout handler was delayed by -1.4371333929999537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:03:11.321732 17273 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.309455 16946 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:11.340792 17154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.332s	user 0.031s	sys 0.060s
I20260709 13:03:11.326684 17258 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:11.349742 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582) took 2486 ms (client timeout 2201 ms). Trace:
W20260709 13:03:11.455971 15231 rpcz_store.cc:269] 0709 13:03:08.863497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.349501 (+2486004us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.349668 (+   167us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:10.983414 15200 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:41710 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714)
W20260709 13:03:11.148211 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 141056us
W20260709 13:03:11.456683 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.372341 15159 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
W20260709 13:03:11.496349 15126 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.932s (SENT)
W20260709 13:03:11.496974 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.456971 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1600) took 2398 ms (client timeout 2012 ms). Trace:
I20260709 13:03:11.421834 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:11.446832 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 116016us
W20260709 13:03:11.498006 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.498023 15231 rpcz_store.cc:269] 0709 13:03:09.058791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.456746 (+2397955us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.456896 (+   150us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.496830 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:11.499287 15051 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.158s (SENT)
W20260709 13:03:11.501060 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1601) took 2362 ms (client timeout 2028 ms). Trace:
W20260709 13:03:11.501549 15232 rpcz_store.cc:269] 0709 13:03:09.138416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.500815 (+2362399us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.500971 (+   156us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.502566 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1602) took 2327 ms (client timeout 2041 ms). Trace:
W20260709 13:03:11.502996 15232 rpcz_store.cc:269] 0709 13:03:09.175138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.502365 (+2327227us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.502505 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.367704 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:11.503754 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 414) took 2197 ms (client timeout 1953 ms). Trace:
W20260709 13:03:11.507367 15232 rpcz_store.cc:269] 0709 13:03:09.306344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.503617 (+2197273us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.503708 (+    91us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.505250 15051 leader_election.cc:336] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.410s (SENT)
I20260709 13:03:11.506942 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:11.509907 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.510155 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:11.505486 15126 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.933s (SENT)
W20260709 13:03:11.511204 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1603) took 2335 ms (client timeout 2086 ms). Trace:
I20260709 13:03:11.511463 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:11.511759 15126 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.934s (SENT)
W20260709 13:03:11.511946 15232 rpcz_store.cc:269] 0709 13:03:09.175605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.510969 (+2335364us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.511122 (+   153us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.338629 17216 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:11.517541 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1569) took 2652 ms (client timeout 2235 ms). Trace:
I20260709 13:03:11.523989 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:11.461684 17154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.032s	sys 0.062s
I20260709 13:03:11.371526 17228 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.371071 17230 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:11.565426 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1698) took 2036 ms (client timeout 1738 ms). Trace:
I20260709 13:03:11.504261 17273 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:11.565393 15231 rpcz_store.cc:269] 0709 13:03:08.853740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.506122 (+2652382us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.506236 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.341741 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.349948 17285 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.460358 17283 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:11.569842 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:11.588228 17228 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.460347 15840 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:11.341953 17246 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:11.459038 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 717) took 2792 ms (client timeout 1516 ms). Trace:
I20260709 13:03:11.339249 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.565618 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:11.572883 15232 rpcz_store.cc:269] 0709 13:03:09.528531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.565003 (+2036472us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.565136 (+   133us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.349188 17140 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:11.392362 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.507602 17260 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:11.589493 15158 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:11.589388 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.589947 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:11.590288 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1604) took 2414 ms (client timeout 2199 ms). Trace:
I20260709 13:03:11.340266 17278 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.591940 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.595996 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.461978 17258 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.564783 17238 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:11.604053 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 258119us
I20260709 13:03:11.506935 17214 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
W20260709 13:03:11.781154 15201 leader_election.cc:336] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.837s (SENT)
I20260709 13:03:11.605559 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.782606 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:11.783108 15080 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:11.784194 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1605) took 2608 ms (client timeout 2211 ms). Trace:
W20260709 13:03:11.784631 15232 rpcz_store.cc:269] 0709 13:03:09.176258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.784006 (+2607748us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.784129 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.785516 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1606) took 2609 ms (client timeout 2235 ms). Trace:
W20260709 13:03:11.786141 15232 rpcz_store.cc:269] 0709 13:03:09.176594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.785361 (+2608767us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.785467 (+   106us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.787037 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1607) took 2610 ms (client timeout 2242 ms). Trace:
I20260709 13:03:11.740262 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.808878 15228 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:11.790814 17278 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.732697 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:11.817505 15085 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.817572 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.819247 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.820551 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:11.821440 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1571) took 2967 ms (client timeout 2086 ms). Trace:
I20260709 13:03:11.821774 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:11.821866 15233 rpcz_store.cc:269] 0709 13:03:08.854225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.821248 (+2967023us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.821373 (+   125us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.822371 15157 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 1.
I20260709 13:03:11.823988 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2183) took 2786 ms. Trace:
I20260709 13:03:11.824365 15088 rpcz_store.cc:276] 0709 13:03:09.037882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.038063 (+   181us) service_pool.cc:224] Handling call
0709 13:03:09.038424 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.260171 (+2221747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.823751 (+563580us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:11.823900 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5108215680}
W20260709 13:03:11.826427 15232 rpcz_store.cc:269] 0709 13:03:09.178988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.786887 (+2607899us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.786986 (+    99us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.574175 16946 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.565075 17216 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:11.673144 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 160756us
I20260709 13:03:11.800405 17246 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:11.597630 17230 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:11.800587 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.864466 15126 leader_election.cc:336] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.994s (SENT)
W20260709 13:03:11.865401 15126 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.012s (SENT)
I20260709 13:03:11.801709 15840 pending_rounds.cc:85] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 5
I20260709 13:03:11.598759 17285 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.605844 17224 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.197s
W20260709 13:03:11.803197 15842 rpcz_store.cc:269] 0709 13:03:08.666230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.670059 (+  3829us) service_pool.cc:224] Handling call
0709 13:03:11.458854 (+2788795us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:11.458964 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5124418816}
I20260709 13:03:11.572624 17106 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:11.594091 17283 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:11.806974 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:11.806434 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 299511us
W20260709 13:03:11.809257 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 415) took 2499 ms (client timeout 1718 ms). Trace:
I20260709 13:03:11.792739 17188 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:11.792966 17233 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.886430 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.784368 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.887216 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2302) took 2703 ms. Trace:
I20260709 13:03:11.888324 15160 rpcz_store.cc:276] 0709 13:03:09.184165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.184285 (+   120us) service_pool.cc:224] Handling call
0709 13:03:09.184546 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.454737 (+2270191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.886967 (+432230us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:11.887158 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4943437696}
W20260709 13:03:11.885154 15234 rpcz_store.cc:269] 0709 13:03:09.310210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.809088 (+2498878us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.809196 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.889745 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.890343 15160 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:11.890609 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:11.883544 15847 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:11.826805 17245 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.636390 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:11.892647 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:11.825809 17202 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.893733 15848 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:11.837325 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 419) took 2465 ms (client timeout 2104 ms). Trace:
W20260709 13:03:11.862140 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.791086 15231 rpcz_store.cc:269] 0709 13:03:09.175931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.585237 (+2409306us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.590216 (+  4979us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:11.870468 15126 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.028s (SENT)
I20260709 13:03:11.867252 17246 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.837069 15156 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:11.871709 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:11.904914 15233 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
W20260709 13:03:11.905555 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1701) took 1769 ms (client timeout 1806 ms). Trace:
W20260709 13:03:11.905980 15233 rpcz_store.cc:269] 0709 13:03:10.135812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.871312 (+1735500us) service_pool.cc:224] Handling call
0709 13:03:11.905487 (+ 34175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.907126 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:11.907853 15233 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:11.605990 17257 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:11.633436 17294 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.909201 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1581) took 3045 ms (client timeout 2159 ms). Trace:
W20260709 13:03:11.909718 15236 rpcz_store.cc:269] 0709 13:03:08.863287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.908974 (+3045687us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.909132 (+   158us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.910475 17298 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:03:11.910939 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:11.912856 17296 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 13:03:11.913383 15229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1594) took 3046 ms (client timeout 1925 ms). Trace:
W20260709 13:03:11.913838 15229 rpcz_store.cc:269] 0709 13:03:08.867119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.913193 (+3046074us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.913314 (+   121us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.914801 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:11.915630 15229 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:11.867720 15840 pending_rounds.cc:107] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:03:11.917124 15840 raft_consensus.cc:2889] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:11.863502 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:11.916503 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:11.895258 15232 rpcz_store.cc:269] 0709 13:03:09.362579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.827201 (+2464622us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:11.827299 (+    98us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:11.868963 17216 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:11.897495 17246 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:11.899317 15237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1572) took 3034 ms (client timeout 1574 ms). Trace:
W20260709 13:03:11.925061 15237 rpcz_store.cc:269] 0709 13:03:08.854493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.091892 (+237399us) service_pool.cc:224] Handling call
0709 13:03:11.888432 (+2796540us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54002d3f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:11.888531 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772864768}
W20260709 13:03:11.902093 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 694) took 3038 ms (client timeout 2837 ms). Trace:
W20260709 13:03:11.902860 17291 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 13:03:11.518903 17178 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.909413 17230 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:11.922573 15229 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:11.896130 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.825892 17140 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:11.903622 17290 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:03:11.927241 15848 rpcz_store.cc:269] 0709 13:03:08.863497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.395539 (+1532042us) service_pool.cc:224] Handling call
0709 13:03:11.886210 (+1490671us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:11.902015 (+ 15805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3013504}
W20260709 13:03:11.965308 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 94872.3us
W20260709 13:03:11.967206 15126 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.041s (SENT)
I20260709 13:03:11.928754 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:11.968081 15126 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.086s (SENT)
I20260709 13:03:11.912585 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:11.887194 17283 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.968448 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.885367 17268 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:11.876924 17220 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:03:11.918879 17204 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:11.934798 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:11.962741 17234 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.004s
I20260709 13:03:11.923297 15841 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:11.978264 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:11.998226 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:11.999039 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1705) took 1026 ms. Trace:
I20260709 13:03:11.999517 15229 rpcz_store.cc:276] 0709 13:03:10.973106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.969088 (+995982us) service_pool.cc:224] Handling call
0709 13:03:11.998976 (+ 29888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.933799 17257 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:11.985687 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:11.978991 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.011173 15848 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:11.996585 15231 raft_consensus.cc:2368] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
W20260709 13:03:12.011785 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 145521us
W20260709 13:03:11.978374 15126 leader_election.cc:336] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.211s (SENT)
I20260709 13:03:12.003515 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.920214 15843 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:11.962899 17117 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.986289 17200 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:11.965278 17285 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:11.924511 17202 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.974436 17277 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:12.022425 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1699) took 1945 ms (client timeout 1951 ms). Trace:
W20260709 13:03:12.034152 15231 rpcz_store.cc:269] 0709 13:03:10.077031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.896063 (+1819032us) service_pool.cc:224] Handling call
0709 13:03:12.022347 (+126284us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:12.016149 15126 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.235s (SENT)
W20260709 13:03:12.007215 15051 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 3.162s (SENT)
I20260709 13:03:12.015954 15081 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:11.977254 17299 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:11.992624 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:11.919373 17233 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.919970 17245 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:11.996040 17278 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.007243 15079 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:12.035280 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:11.987102 17268 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:11.995832 17204 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.010895 17178 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:12.043941 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149224us
I20260709 13:03:12.006474 17106 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:12.067269 17106 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:11.909446 17294 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.070497 17294 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:12.071856 17115 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.000s
I20260709 13:03:12.074239 17233 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.075642 15161 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:12.072443 17115 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.076808 15082 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:12.077840 17115 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.045365 15156 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:11.994807 17302 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:03:12.058122 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.035279 15236 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.065788 17204 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:12.065589 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49469.7us
W20260709 13:03:12.120267 15126 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.242s (SENT)
I20260709 13:03:12.120741 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:12.045661 17310 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.122388 17310 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.064939 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1712) took 1089 ms. Trace:
I20260709 13:03:12.125488 15233 rpcz_store.cc:276] 0709 13:03:10.975106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.908787 (+933681us) service_pool.cc:224] Handling call
0709 13:03:12.062297 (+153510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17152}
I20260709 13:03:12.126921 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.127977 15233 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:12.084385 17187 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.000s
I20260709 13:03:12.129274 17187 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.132402 17187 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:12.087164 17295 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.214s	user 0.002s	sys 0.000s
I20260709 13:03:12.087951 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2190) took 3046 ms. Trace:
I20260709 13:03:12.088207 15154 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:03:12.088773 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1565) took 3247 ms (client timeout 2028 ms). Trace:
W20260709 13:03:12.146317 15235 rpcz_store.cc:269] 0709 13:03:08.841324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.088588 (+3247264us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:12.088709 (+   121us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:12.031325 17117 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.005740 17234 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.089607 17268 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.090618 17307 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:12.091495 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.091781 15840 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:12.092166 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 723) took 3424 ms. Trace:
I20260709 13:03:12.092769 17245 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.156363 15845 rpcz_store.cc:276] 0709 13:03:08.667700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.792351 (+124651us) service_pool.cc:224] Handling call
0709 13:03:08.792703 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.816856 (+2024153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.091922 (+1275066us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.092102 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3286480768}
W20260709 13:03:12.092921 15230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1573) took 3238 ms (client timeout 1589 ms). Trace:
W20260709 13:03:12.094439 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.157533 15230 rpcz_store.cc:269] 0709 13:03:08.854784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.092705 (+3237921us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:12.092826 (+   121us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:12.158349 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.158452 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.093755 17257 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.084688 15085 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:12.034225 17200 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.163758 17200 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.166687 17200 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.057833 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.047446 15086 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:12.171284 15086 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:12.046906 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2303) took 2862 ms. Trace:
I20260709 13:03:12.047874 17228 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.084949 17277 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:12.133102 15088 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:12.138520 15083 rpcz_store.cc:276] 0709 13:03:09.041423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.914362 (+872939us) service_pool.cc:224] Handling call
0709 13:03:09.914679 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.225200 (+1310521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.087700 (+862500us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540029f9d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.087890 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012117376}
I20260709 13:03:12.147039 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1703) took 1305 ms. Trace:
I20260709 13:03:12.146988 17310 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.120671 17298 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:12.156263 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.177413 17268 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.185935 15086 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.152814 17115 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:03:12.209836 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89607.5us
W20260709 13:03:12.147535 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105392us
W20260709 13:03:12.154554 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.160646 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:12.102219 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 98822us
I20260709 13:03:12.172142 15163 rpcz_store.cc:276] 0709 13:03:09.184787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.184951 (+   164us) service_pool.cc:224] Handling call
0709 13:03:09.212659 (+ 27708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.150654 (+1937995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.046613 (+895959us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54003ffcd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.046831 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4348975488}
W20260709 13:03:12.211000 15202 connection.cc:256] RPC call timeout handler was delayed by -1.8028201880000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 13:03:12.174029 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.211071 15230 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:12.211540 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.212005 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.212317 15202 connection.cc:256] RPC call timeout handler was delayed by -1.7128201880000233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:03:12.175417 15159 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:12.214145 15086 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:12.222290 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.222759 15202 connection.cc:256] RPC call timeout handler was delayed by -1.5490201879999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 13:03:12.223464 17208 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.195317 15079 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:12.225832 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:12.224372 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.227650 15163 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:12.184223 17302 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:12.207465 17140 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:12.223133 15202 connection.cc:256] RPC call timeout handler was delayed by -1.4797201879999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 13:03:12.237432 15202 connection.cc:256] RPC call timeout handler was delayed by -1.4689201880000837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
I20260709 13:03:12.229693 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:12.238147 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.224231 17228 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.238617 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.183341 15236 rpcz_store.cc:276] 0709 13:03:10.841479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.912202 (+1070723us) service_pool.cc:224] Handling call
0709 13:03:12.138129 (+225927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:12.233820 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:12.190469 17310 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:12.229724 17293 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:03:12.199484 17313 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:12.251036 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.251943 15230 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:12.238837 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.253423 15160 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
W20260709 13:03:12.254343 15810 connection.cc:256] RPC call timeout handler was delayed by -2.0487004430000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:03:12.254962 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.237818 15202 connection.cc:256] RPC call timeout handler was delayed by -1.2457201880000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
W20260709 13:03:12.255362 15202 connection.cc:256] RPC call timeout handler was delayed by -1.1422201880000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22810000000000002s for a response
I20260709 13:03:12.255566 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.256071 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:12.244145 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 724) took 3576 ms. Trace:
W20260709 13:03:12.248149 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.249111 15235 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.228876 15233 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.240025 15163 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:12.254658 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.227955 17187 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:03:12.266692 15052 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.516s (SENT)
W20260709 13:03:12.261363 15810 connection.cc:256] RPC call timeout handler was delayed by -1.7346004429999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
I20260709 13:03:12.263589 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:12.259473 15844 rpcz_store.cc:276] 0709 13:03:08.667890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.832069 (+164179us) service_pool.cc:224] Handling call
0709 13:03:08.857360 (+ 25291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.925667 (+1068307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.243800 (+2318133us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400730354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.244064 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270865664}
I20260709 13:03:12.268452 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:12.268878 15810 connection.cc:256] RPC call timeout handler was delayed by -1.5825004429999581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:03:12.269335 15810 connection.cc:256] RPC call timeout handler was delayed by -1.547400443000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 13:03:12.269938 15810 connection.cc:256] RPC call timeout handler was delayed by -1.4276687390000689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 13:03:12.271358 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:12.274359 15810 connection.cc:256] RPC call timeout handler was delayed by -1.2161687390000679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
I20260709 13:03:12.264221 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.276343 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:12.276978 17220 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.277470 17220 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.277750 17117 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.278759 17311 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:12.268443 17273 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.005s
I20260709 13:03:12.279517 17220 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:12.279774 17273 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.280068 17220 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:12.273622 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:12.281791 15846 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.244287 17298 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.285290 15229 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:12.285856 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:12.286139 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 698) took 2328 ms. Trace:
I20260709 13:03:12.286722 15846 rpcz_store.cc:276] 0709 13:03:09.957301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.990770 (+1033469us) service_pool.cc:224] Handling call
0709 13:03:12.281431 (+1290661us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:12.281636 (+   205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.282181 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.286087 (+  3906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4921088}
I20260709 13:03:12.284749 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1707) took 1311 ms. Trace:
I20260709 13:03:12.292958 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.293819 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:12.295547 15086 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:12.266812 17009 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.296125 15086 pending_rounds.cc:85] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 5
I20260709 13:03:12.268205 17294 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:12.228215 17208 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.273058 17140 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.283610 17035 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:12.281350 17257 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:12.303159 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:12.285218 17188 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:12.299661 15086 pending_rounds.cc:107] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:03:12.250648 17106 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:03:12.290365 17283 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.316046 17234 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.256338 17228 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.297101 17238 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:12.302057 17224 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:12.294268 17311 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:12.318431 15080 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:12.311950 15086 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:12.329945 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:12.328647 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65209.8us
I20260709 13:03:12.330667 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.299665 17233 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.306433 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.297452 17318 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:12.316821 17283 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.304119 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:12.326526 15229 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:12.341900 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43597.4us
I20260709 13:03:12.317617 17230 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:12.295019 15233 rpcz_store.cc:276] 0709 13:03:10.973731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.128782 (+1155051us) service_pool.cc:224] Handling call
0709 13:03:12.284673 (+155891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:12.309679 17140 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.305366 17009 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.307278 17035 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.347853 17311 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:12.348548 17035 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:12.348730 17311 consensus_queue.cc:1243] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:03:12.349324 17311 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:12.349750 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.349939 17311 consensus_queue.cc:1243] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:12.350383 15087 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:12.351787 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:12.352465 15087 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:12.343722 16946 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:03:12.353286 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:12.353588 17315 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:12.354113 15233 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.297245 17298 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.310130 17220 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:12.329428 17188 raft_consensus.cc:2764] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:03:12.355325 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.353334 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 582) took 1033 ms. Trace:
I20260709 13:03:12.358134 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.364125 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.364712 15233 raft_consensus.cc:2368] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:12.364869 15082 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:12.366150 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.366526 15082 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.358155 17257 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.335611 17234 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.363314 15084 rpcz_store.cc:276] 0709 13:03:11.319600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.319783 (+   183us) service_pool.cc:224] Handling call
0709 13:03:11.332738 (+ 12955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.565844 (+233106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.565872 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.565913 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:11.565947 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.353119 (+787172us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540014df54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.353159 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.353285 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535593216}
I20260709 13:03:12.347694 17283 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:03:12.345060 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:12.344314 17233 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:12.347666 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59966.7us
I20260709 13:03:12.374414 15086 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:12.383451 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:12.346030 17230 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.384989 17230 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:12.386786 15126 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.610s (SENT)
W20260709 13:03:12.388187 15126 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.650s (SENT)
W20260709 13:03:12.389237 15126 leader_election.cc:336] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.813s (SENT)
W20260709 13:03:12.389477 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.383780 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:12.368810 15087 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:12.389735 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:12.390811 17232 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:12.392195 15087 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:12.392999 15079 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:12.394040 17273 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.394796 17230 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.397215 17246 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.358270 17224 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:12.398581 17188 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:12.398702 17224 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:12.399039 17188 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.399420 17188 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:12.399572 17246 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.399755 17224 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.381641 17234 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.408800 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.409572 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:12.413295 15126 leader_election.cc:336] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.959s (SENT)
W20260709 13:03:12.415879 15126 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.996s (SENT)
I20260709 13:03:12.420482 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.368152 17257 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:12.425035 15231 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:12.375473 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.430220 17009 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.430945 17220 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:12.442072 17257 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.442768 17220 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:12.443269 17222 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:12.443485 17257 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.443733 17222 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:12.444284 17316 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:03:12.404247 17140 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.431546 17304 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.441062 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.441421 17298 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:12.403082 17234 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.400806 17224 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:12.478099 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.484550 15160 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:12.486274 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.486704 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.443807 17220 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.373675 15233 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:12.446359 15086 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:12.480530 15845 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:12.498196 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:12.481559 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.478056 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.499528 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:12.499958 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:12.482352 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 99811.3us
I20260709 13:03:12.490473 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:12.503098 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.504312 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.471822 17307 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.401667 17216 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:12.508594 15228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1577) took 3647 ms (client timeout 1533 ms). Trace:
I20260709 13:03:12.525902 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:12.523065 15228 rpcz_store.cc:269] 0709 13:03:08.862424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.880664 (+ 18240us) service_pool.cc:224] Handling call
0709 13:03:12.508424 (+3627760us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54000ad7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:12.508530 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5569729408}
I20260709 13:03:12.422672 17035 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.476915 17314 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.432350 17188 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.493151 17230 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:12.539016 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:12.540354 15160 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:12.481724 17268 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:03:12.505383 17106 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:12.509315 17298 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.543542 15846 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5.
I20260709 13:03:12.544605 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.545292 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.546530 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:12.484177 17323 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:03:12.547534 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.510778 17228 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.521548 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:12.553830 17285 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:12.554584 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:12.651033 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.651773 15161 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:12.555217 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.555542 15084 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.557777 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.653163 15229 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.654040 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.655004 15229 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.558597 17245 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:12.564816 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:12.570523 17327 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:12.550634 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.543967 17246 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:12.623857 17178 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:12.661810 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.666571 15229 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:12.667912 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.669411 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.572520 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.670619 15237 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:12.538444 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.507588 17234 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:03:12.649654 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 103171us
I20260709 13:03:12.656737 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:12.659271 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1706) took 1685 ms (client timeout 1979 ms). Trace:
I20260709 13:03:12.545122 17320 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.664037 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.561265 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.482053 17304 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:12.690725 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 140821us
I20260709 13:03:12.673046 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2313) took 3317 ms. Trace:
I20260709 13:03:12.699254 15162 rpcz_store.cc:276] 0709 13:03:09.356228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.918624 (+562396us) service_pool.cc:224] Handling call
0709 13:03:09.939070 (+ 20446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.219628 (+1280558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.672829 (+1453201us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002882d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:12.672987 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2943623040}
W20260709 13:03:12.673084 15237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1716) took 1697 ms (client timeout 1924 ms). Trace:
I20260709 13:03:12.554630 17308 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:03:12.679668 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.521275 17329 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 13:03:12.643620 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.684612 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:12.662501 17258 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.833s	user 0.020s	sys 0.000s
I20260709 13:03:12.655802 17285 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:12.643693 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:12.682682 15234 rpcz_store.cc:269] 0709 13:03:10.973458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.926343 (+952885us) service_pool.cc:224] Handling call
0709 13:03:12.555101 (+628758us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfe9c 00007f0d3c8bfa5b 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:12.659189 (+104088us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18616448}
I20260709 13:03:12.717247 15162 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:12.529352 17233 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.527160 17307 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.649173 17330 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:12.571960 17273 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:03:12.770506 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.622828 17228 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:12.658044 17245 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:12.661593 17246 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.573346 17298 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.801932 17245 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.802943 17298 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:12.803125 17245 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:12.803478 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.804102 17298 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:12.804273 15230 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.706176 15161 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:12.691910 15079 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:12.768275 17323 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:12.709525 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.510849 17222 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.709977 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.714720 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:12.694768 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:12.828900 15845 raft_consensus.cc:2410] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:12.829093 15160 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:03:12.731208 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:12.704905 15237 rpcz_store.cc:269] 0709 13:03:10.976092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.926135 (+950043us) service_pool.cc:224] Handling call
0709 13:03:12.572344 (+646209us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:12.672964 (+100620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18335104}
I20260709 13:03:12.837630 17222 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:12.837883 17321 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:03:12.840098 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.840417 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:12.840935 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 126318us
I20260709 13:03:12.841535 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:12.807106 17246 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:12.698146 17009 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:12.541288 17314 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.718675 15086 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:12.771193 15234 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:12.705085 17308 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:12.882784 17308 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.884141 17308 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:12.734489 17278 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.671s	user 0.016s	sys 0.004s
I20260709 13:03:12.884977 17326 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.885296 17278 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:12.886188 15157 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:12.885779 17326 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.733722 15233 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:12.828662 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.887199 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1731) took 1090 ms. Trace:
I20260709 13:03:12.887626 15233 rpcz_store.cc:276] 0709 13:03:11.796285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.539606 (+743321us) service_pool.cc:224] Handling call
0709 13:03:12.887118 (+347512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:12.888092 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:12.766645 17268 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.713125 17328 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.355s
I20260709 13:03:12.740993 17258 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.796073 17330 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:12.846743 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:12.847434 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:12.837667 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:12.881548 17009 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:12.842211 17285 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.798326 17228 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:12.804581 17298 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.706271 17320 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.808351 15236 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:12.969523 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.970750 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.971275 15081 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:12.923296 17035 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:12.932682 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 86051.4us
I20260709 13:03:12.889204 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:12.906213 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58782.2us
W20260709 13:03:12.732429 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.742566 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 177823us
I20260709 13:03:12.942602 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:12.889669 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:12.733057 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 186600us
W20260709 13:03:12.890342 17333 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:03:12.958722 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:12.892058 17341 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:12.843927 17053 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.888732 17326 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:12.893069 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:12.941661 17216 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:13.046670 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:12.972141 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:13.045857 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 407) took 1674 ms (client timeout 1843 ms). Trace:
I20260709 13:03:13.421818 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1746) took 1061 ms. Trace:
I20260709 13:03:13.253124 15230 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:13.422446 15163 rpcz_store.cc:276] 0709 13:03:12.360384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.360945 (+   561us) service_pool.cc:224] Handling call
0709 13:03:13.046538 (+685593us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:13.421744 (+375206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3647744}
I20260709 13:03:12.892365 15154 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:13.423321 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 422) took 1187 ms. Trace:
I20260709 13:03:13.424773 15163 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:13.422281 15159 rpcz_store.cc:269] 0709 13:03:11.371390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.504916 (+133526us) service_pool.cc:224] Handling call
0709 13:03:13.045664 (+1540748us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54001c5cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:13.045789 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891914496}
I20260709 13:03:12.956279 17188 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.426436 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 427) took 1166 ms. Trace:
I20260709 13:03:13.148694 15841 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 2: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:13.427205 15053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.000s	sys 0.016s
W20260709 13:03:13.417269 17294 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.110s	user 0.012s	sys 0.000s
W20260709 13:03:13.422597 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 450504us
W20260709 13:03:13.424729 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.012s	sys 0.008s
W20260709 13:03:13.497129 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.013s	sys 0.008s
I20260709 13:03:13.501263 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:13.277885 17337 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.433526 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2322) took 1062 ms. Trace:
W20260709 13:03:13.437608 17245 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.006s
W20260709 13:03:13.404866 17204 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.267s	user 0.009s	sys 0.005s
W20260709 13:03:13.430152 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.513247 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.946870 17009 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.435766 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:13.516906 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 81167.7us
I20260709 13:03:13.517745 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:13.437436 15053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.000s	sys 0.017s
I20260709 13:03:13.279858 17320 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:13.519704 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.433799 15230 rpcz_store.cc:276] 0709 13:03:12.236305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.803087 (+566782us) service_pool.cc:224] Handling call
0709 13:03:13.423267 (+620180us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:13.151198 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.960712 17298 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:13.362931 17228 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:13.353346 17224 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.857s	user 0.008s	sys 0.003s
W20260709 13:03:13.415588 17257 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.942s	user 0.009s	sys 0.001s
I20260709 13:03:13.438350 17053 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:12.947989 17268 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.123263 17314 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.473487 15085 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.511198 17115 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:13.473892 15233 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:13.424597 17285 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:13.436470 15237 rpcz_store.cc:276] 0709 13:03:12.259993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.934274 (+674281us) service_pool.cc:224] Handling call
0709 13:03:13.426373 (+492099us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.473487 17117 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:13.330721 17330 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.037719 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1613) took 1310 ms. Trace:
I20260709 13:03:13.476099 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 691) took 1119 ms. Trace:
I20260709 13:03:13.525296 17188 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.476621 17140 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:13.477402 16946 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.426671 17329 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.477840 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:13.558943 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:13.559847 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 82109.3us
I20260709 13:03:13.559864 15237 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:13.560626 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1739) took 1762 ms. Trace:
I20260709 13:03:13.560979 15237 rpcz_store.cc:276] 0709 13:03:11.798169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.542598 (+1744429us) service_pool.cc:224] Handling call
0709 13:03:13.560570 (+ 17972us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.480743 15160 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.485106 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:13.214231 17327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.013s	sys 0.023s
I20260709 13:03:13.111814 15842 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.423837 17258 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:13.426584 17331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.002s	sys 0.017s
I20260709 13:03:13.428884 15234 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:13.508533 15163 rpcz_store.cc:276] 0709 13:03:12.370702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.423825 (+1053123us) service_pool.cc:224] Handling call
0709 13:03:13.424201 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.433242 (+  9041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.433464 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.228966 17338 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:13.520166 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.521042 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:13.498728 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:13.518538 17328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.025s	sys 0.028s
I20260709 13:03:13.508805 15051 leader_election.cc:400] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:13.476119 17297 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:13.427093 15128 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:13.210058 17346 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.533316 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2194) took 1276 ms. Trace:
I20260709 13:03:13.466473 17238 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:13.549616 17304 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:13.540699 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.510234 17294 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.516414 17154 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.522778 17245 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.523283 17345 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.549584 15844 rpcz_store.cc:276] 0709 13:03:12.356524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.356682 (+   158us) service_pool.cc:224] Handling call
0709 13:03:12.367248 (+ 10566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.379271 (+ 12023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.379303 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.379982 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.438789 (+ 58807us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:12.438846 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.439521 (+   675us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.475842 (+1036321us) raft_consensus.cc:1685] finished
0709 13:03:13.475866 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.476030 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10156,"raft.queue_time_us":39535,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7401,"thread_start_us":58366,"threads_started":1,"wal-append.queue_time_us":148939}
W20260709 13:03:13.551877 17097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.019s	sys 0.022s
I20260709 13:03:13.557531 16946 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.532275 17228 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:03:13.630856 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 145834us
I20260709 13:03:13.597757 15230 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:13.563376 15229 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:13.632046 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.563618 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:13.540339 17115 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.549191 15236 rpcz_store.cc:276] 0709 13:03:11.676233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.267070 (+590837us) service_pool.cc:224] Handling call
0709 13:03:12.986837 (+719767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.633697 17115 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:13.518832 17204 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.591341 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:13.636114 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1622) took 1145 ms. Trace:
I20260709 13:03:13.545627 17117 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:13.540081 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2312) took 4212 ms. Trace:
I20260709 13:03:13.556579 17140 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:13.564319 17327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.023s	sys 0.040s
W20260709 13:03:13.589811 17331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.004s	sys 0.020s
I20260709 13:03:13.614228 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1724) took 2361 ms. Trace:
W20260709 13:03:13.589757 17356 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:03:13.599557 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 172570us
I20260709 13:03:13.666322 17348 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:13.592175 17353 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:13.639025 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 600) took 1129 ms. Trace:
I20260709 13:03:13.667748 15079 rpcz_store.cc:276] 0709 13:03:12.510157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.907249 (+397092us) service_pool.cc:224] Handling call
0709 13:03:12.968913 (+ 61664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.112777 (+143864us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:13.437345 (+324568us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.638594 (+201249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.638653 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:13.638706 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.638794 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.638959 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":269420,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2504,"raft.queue_time_us":136,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13828,"thread_start_us":119489,"threads_started":1,"wal-append.queue_time_us":216211}
I20260709 13:03:13.636197 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 750) took 1134 ms. Trace:
W20260709 13:03:13.598995 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 94424us
W20260709 13:03:13.595270 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.479195 17216 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:13.517481 17337 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.559458 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1620) took 1068 ms. Trace:
W20260709 13:03:13.633409 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 69908.7us
I20260709 13:03:13.634407 17245 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.679046 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:13.558627 15232 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:13.636561 15237 rpcz_store.cc:276] 0709 13:03:12.490331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.593954 (+1103623us) service_pool.cc:224] Handling call
0709 13:03:13.636043 (+ 42089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.636566 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.636566 15163 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.637748 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.637739 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1725) took 2384 ms. Trace:
I20260709 13:03:13.682612 15230 rpcz_store.cc:276] 0709 13:03:11.253612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.520764 (+2267152us) service_pool.cc:224] Handling call
0709 13:03:13.637683 (+116919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.615506 17314 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:13.594587 17338 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.691201 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:13.662351 17365 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:13.696538 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:13.653716 15158 rpcz_store.cc:276] 0709 13:03:09.327901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.524369 (+196468us) service_pool.cc:224] Handling call
0709 13:03:09.524712 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.787989 (+2263277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.539806 (+1751817us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54003e0b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:13.540017 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4748125568}
I20260709 13:03:13.634637 17154 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.655512 17345 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.690845 17117 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.703264 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:13.677484 15843 rpcz_store.cc:276] 0709 13:03:12.502005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.505200 (+  3195us) service_pool.cc:224] Handling call
0709 13:03:13.481963 (+976763us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:13.636136 (+154173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8665344}
I20260709 13:03:13.704070 15082 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:13.658061 17140 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:13.658651 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.613404 17297 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:13.630611 17097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.024s	sys 0.024s
I20260709 13:03:13.634361 17328 consensus_queue.cc:1243] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:03:13.629925 17238 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:13.663692 15159 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 3. Ops: []
I20260709 13:03:13.638106 17304 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:13.615124 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.714785 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.715348 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.715730 17370 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:13.710594 17053 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.661789 15234 rpcz_store.cc:276] 0709 13:03:11.253271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.908282 (+1655011us) service_pool.cc:224] Handling call
0709 13:03:13.614133 (+705851us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.679111 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:13.682054 15848 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:13.600303 15085 rpcz_store.cc:276] 0709 13:03:12.256872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.257479 (+   607us) service_pool.cc:224] Handling call
0709 13:03:12.257727 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.465535 (+1207808us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b54001c2b20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:03:13.532934 (+ 67399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.533239 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15972224}
I20260709 13:03:13.668574 17348 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.669006 17320 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.721926 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:13.698778 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.701507 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:13.694087 17338 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.693874 17331 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:13.695446 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.682054 15233 rpcz_store.cc:276] 0709 13:03:12.488713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.888439 (+399726us) service_pool.cc:224] Handling call
0709 13:03:13.557027 (+668588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.698745 17314 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:13.699671 17329 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:13.712695 15201 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.931s (SENT)
I20260709 13:03:13.725497 15158 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:13.743269 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:13.743464 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:13.710950 17337 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:13.730420 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:13.735386 15236 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:13.726193 17326 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.736510 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:13.739030 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:13.732739 17327 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.743875 15158 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:13.716248 17370 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:13.746984 17329 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.717682 17140 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:13.750222 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:13.735797 17297 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.737809 17314 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:13.752130 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1619) took 1263 ms. Trace:
I20260709 13:03:13.730067 17097 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.773207 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1728) took 2457 ms. Trace:
I20260709 13:03:13.773619 15155 rpcz_store.cc:276] 0709 13:03:11.315635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.330568 (+ 14933us) service_pool.cc:224] Handling call
0709 13:03:13.046061 (+1715493us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:13.046274 (+   213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.114036 (+ 67762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.114088 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.429874 (+315786us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.598758 (+168884us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:13.598806 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.598881 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.772909 (+174028us) raft_consensus.cc:1685] finished
0709 13:03:13.772928 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.773143 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28214,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":144098,"spinlock_wait_cycles":19616512,"thread_start_us":125067,"threads_started":1,"wal-append.queue_time_us":253964}
I20260709 13:03:13.775300 17370 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:13.775894 17370 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.776043 15155 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.776417 17370 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.764168 17238 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.732451 17320 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.753168 17053 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.752133 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.740418 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.728519 17228 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.782685 17364 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:13.783155 17364 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.783075 17228 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.783696 17364 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.738257 17331 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.704545 17154 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.742508 17268 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.715090 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:13.770253 17354 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:13.770411 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:13.771458 17366 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:03:13.797295 17246 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.002s
I20260709 13:03:13.797832 17246 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.800102 17246 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.779320 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:13.800882 17371 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:13.801416 17371 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.769752 15236 rpcz_store.cc:276] 0709 13:03:12.488475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.664988 (+1176513us) service_pool.cc:224] Handling call
0709 13:03:13.752061 (+ 87073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.804287 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.776731 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2326) took 1394 ms. Trace:
I20260709 13:03:13.786206 17053 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.805244 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 423) took 1568 ms. Trace:
I20260709 13:03:13.805490 17285 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:13.753240 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:13.805895 17285 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.806769 17285 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:13.733373 17348 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.808421 15158 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:13.756803 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.816516 15842 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:13.778342 17238 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.785720 15845 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:13.788579 17331 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:13.786422 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:13.785183 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:13.755254 17326 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:13.829957 17323 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.002s	user 0.004s	sys 0.008s
I20260709 13:03:13.830425 17323 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:13.798233 17106 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.831234 17106 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:13.833835 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:13.804965 15155 rpcz_store.cc:276] 0709 13:03:12.384340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.775599 (+1391259us) service_pool.cc:224] Handling call
0709 13:03:13.775893 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.776500 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.776683 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.835546 15155 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.836277 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2328) took 1300 ms. Trace:
I20260709 13:03:13.836679 15155 rpcz_store.cc:276] 0709 13:03:12.535958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.835103 (+1299145us) service_pool.cc:224] Handling call
0709 13:03:13.835372 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.835981 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.836211 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.782075 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1615) took 1294 ms. Trace:
I20260709 13:03:13.837953 15234 rpcz_store.cc:276] 0709 13:03:12.487579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.718418 (+1230839us) service_pool.cc:224] Handling call
0709 13:03:13.782003 (+ 63585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.840102 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:13.810046 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:13.784121 17008 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:13.805711 15236 rpcz_store.cc:276] 0709 13:03:12.236700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.804001 (+1567301us) service_pool.cc:224] Handling call
0709 13:03:13.805175 (+  1174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.762564 17329 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.785183 17009 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.818010 17371 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.818091 17320 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.868468 17218 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:13.869037 17218 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:13.846629 17268 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:13.869957 17218 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.814435 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.783780 17202 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.872816 15237 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:13.761587 17346 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:13.825181 17245 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:13.874838 17009 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:13.800657 17357 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:03:13.876227 17285 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.876899 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:13.877563 15848 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:13.877753 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 747) took 1377 ms. Trace:
I20260709 13:03:13.825891 17307 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:13.857550 17369 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:13.865067 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.885429 15234 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:13.856292 17115 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.829191 17228 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.835179 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:13.821715 17370 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:13.846879 17008 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:13.893942 17369 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:13.894434 17008 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.894984 17369 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.895594 17008 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:13.897176 15051 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.979s (SENT)
I20260709 13:03:13.884402 15843 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:13.901273 17345 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:03:13.821449 17354 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:13.869494 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:13.969969 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2329) took 1430 ms. Trace:
I20260709 13:03:14.020653 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:13.963941 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:14.055390 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.041225 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 77377.6us
W20260709 13:03:13.935977 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.052232 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:13.993531 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.050365 15234 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:13.962391 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 152448us
W20260709 13:03:14.059525 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.963670 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:13.911263 15051 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.136s (SENT)
I20260709 13:03:14.040683 15155 rpcz_store.cc:276] 0709 13:03:12.539227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.837680 (+1298453us) service_pool.cc:224] Handling call
0709 13:03:13.965602 (+127922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.965836 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.965872 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.965926 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:13.965982 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.966057 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.969417 (+  3360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.060484 15228 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:14.063247 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1623) took 1572 ms. Trace:
W20260709 13:03:14.060134 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.913986 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2201) took 1650 ms. Trace:
W20260709 13:03:14.060701 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 149481us
I20260709 13:03:14.060997 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:14.059580 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1747) took 1085 ms. Trace:
I20260709 13:03:14.076514 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:14.076514 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1750) took 1654 ms. Trace:
I20260709 13:03:14.080104 15084 rpcz_store.cc:276] 0709 13:03:12.263848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.368575 (+104727us) service_pool.cc:224] Handling call
0709 13:03:12.555273 (+186698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.652470 (+ 97197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.913752 (+1261282us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540014e450. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:13.913914 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395094912}
W20260709 13:03:14.080931 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.876274 17298 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:14.083840 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.081300 15234 rpcz_store.cc:276] 0709 13:03:12.974255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.963387 (+989132us) service_pool.cc:224] Handling call
0709 13:03:14.059527 (+ 96140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:14.091627 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.063891 15228 rpcz_store.cc:276] 0709 13:03:12.490564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.963416 (+1472852us) service_pool.cc:224] Handling call
0709 13:03:14.063187 (+ 99771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.034816 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1736) took 2058 ms. Trace:
I20260709 13:03:13.961601 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 686) took 2457 ms. Trace:
I20260709 13:03:14.081900 15155 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:14.364141 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:14.127322 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:14.019611 17218 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:14.769098 17338 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.001s
W20260709 13:03:14.817157 17380 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.608s	user 0.001s	sys 0.000s
W20260709 13:03:14.725288 17330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.026s	sys 0.034s
W20260709 13:03:14.823132 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.881348 15840 rpcz_store.cc:276] 0709 13:03:12.500576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.505114 (+  4538us) service_pool.cc:224] Handling call
0709 13:03:13.876688 (+1371574us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:13.877688 (+  1000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7058304}
W20260709 13:03:14.826969 15126 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.200s (SENT)
I20260709 13:03:14.978233 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:14.825919 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 592159us
W20260709 13:03:14.825728 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 679869us
W20260709 13:03:14.822188 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1740) took 2948 ms (client timeout 1979 ms). Trace:
W20260709 13:03:14.980700 15232 rpcz_store.cc:269] 0709 13:03:11.873375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.497967 (+624592us) service_pool.cc:224] Handling call
0709 13:03:14.821981 (+2324014us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400057dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:14.822108 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379068672}
W20260709 13:03:14.829474 17106 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.997s	user 0.010s	sys 0.002s
I20260709 13:03:14.081137 15163 rpcz_store.cc:276] 0709 13:03:12.422085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.833183 (+1411098us) service_pool.cc:224] Handling call
0709 13:03:13.833507 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.833895 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.833931 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.834394 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.966690 (+132296us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:13.966746 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.966852 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.076206 (+109354us) raft_consensus.cc:1685] finished
0709 13:03:14.076235 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.076436 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1775,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":12990,"thread_start_us":15261,"threads_started":1,"wal-append.queue_time_us":239317}
I20260709 13:03:14.982317 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:14.823994 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:14.034767 17326 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:14.962064 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 433) took 1094 ms. Trace:
W20260709 13:03:14.961552 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 138574us
I20260709 13:03:14.988250 15084 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:14.988098 15155 rpcz_store.cc:276] 0709 13:03:13.867361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.060742 (+193381us) service_pool.cc:224] Handling call
0709 13:03:14.961982 (+901240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.989044 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2211) took 1242 ms. Trace:
I20260709 13:03:14.034081 17115 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:14.298132 15051 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.171s (SENT)
I20260709 13:03:15.026132 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:15.026980 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:15.027279 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47039.9us
I20260709 13:03:15.027942 15847 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:14.038743 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:14.971189 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:15.029439 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48010.8us
I20260709 13:03:15.029726 15228 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:14.052525 17355 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:14.970605 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.030638 15234 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:14.002821 17214 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.031143 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1751) took 2005 ms. Trace:
I20260709 13:03:15.031453 17214 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.031708 15234 rpcz_store.cc:276] 0709 13:03:13.025799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.823623 (+1797824us) service_pool.cc:224] Handling call
0709 13:03:15.031051 (+207428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.033560 15234 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.034228 17214 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.034407 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1575) took 6178 ms. Trace:
W20260709 13:03:14.819839 17329 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.966s	user 0.003s	sys 0.006s
I20260709 13:03:14.119560 15236 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:15.035626 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1745) took 2062 ms (client timeout 2326 ms). Trace:
W20260709 13:03:15.036247 15236 rpcz_store.cc:269] 0709 13:03:12.973614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.869165 (+895551us) service_pool.cc:224] Handling call
0709 13:03:15.035555 (+1166390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.953313 17304 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:14.982489 15842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.025s	sys 0.032s
I20260709 13:03:14.347935 17354 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:14.982935 15163 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:14.983633 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.044s	sys 0.020s
W20260709 13:03:14.630721 17268 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.761s	user 0.007s	sys 0.004s
I20260709 13:03:14.989544 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:14.989383 15084 rpcz_store.cc:276] 0709 13:03:13.746443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.970469 (+1224026us) service_pool.cc:224] Handling call
0709 13:03:14.988075 (+ 17606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.988763 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.988987 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:14.991199 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.458451 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.343185 17364 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 13:03:15.035353 17329 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:13.912761 17208 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:15.066659 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.035012 15234 rpcz_store.cc:276] 0709 13:03:08.860931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.032822 (+6171891us) service_pool.cc:224] Handling call
0709 13:03:15.033306 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.034071 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.034332 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.038717 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1755) took 2011 ms. Trace:
I20260709 13:03:15.068351 15228 rpcz_store.cc:276] 0709 13:03:13.027053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.824292 (+1797239us) service_pool.cc:224] Handling call
0709 13:03:15.038661 (+214369us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:14.975983 17330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.026s	sys 0.034s
I20260709 13:03:15.038358 17355 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:15.048558 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.045s	sys 0.020s
I20260709 13:03:15.048595 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 435) took 1180 ms. Trace:
I20260709 13:03:14.973584 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 693) took 1354 ms. Trace:
W20260709 13:03:14.989737 17314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.003s	sys 0.010s
I20260709 13:03:15.025719 15232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:57414
I20260709 13:03:15.070477 15163 rpcz_store.cc:276] 0709 13:03:13.867973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.982078 (+1114105us) service_pool.cc:224] Handling call
0709 13:03:15.048512 (+ 66434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.071110 15232 raft_consensus.cc:606] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:15.071475 15232 raft_consensus.cc:614] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:15.072245 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 426) took 1540 ms. Trace:
W20260709 13:03:15.073063 17314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.004s	sys 0.010s
I20260709 13:03:15.073298 17298 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.074110 15848 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:15.074613 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.075253 15840 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:15.052253 17115 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:15.075874 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 722) took 1012 ms. Trace:
I20260709 13:03:14.076073 17331 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.052790 15155 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.053222 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 588) took 3077 ms. Trace:
W20260709 13:03:15.046216 15842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.025s	sys 0.032s
W20260709 13:03:15.067720 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.901963 15233 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:14.975454 17338 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.069231 17222 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.072602 15163 rpcz_store.cc:276] 0709 13:03:13.531445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.071385 (+1539940us) service_pool.cc:224] Handling call
0709 13:03:15.071690 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.071889 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.071924 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.071971 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.072016 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.072084 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.072195 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.108762 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.109323 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.053571 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.073912 17139 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.952s
I20260709 13:03:15.083662 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57414 (request call id 17) took 1275 ms. Trace:
W20260709 13:03:15.110025 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.080804 17208 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:15.085258 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1744) took 2112 ms (client timeout 2215 ms). Trace:
I20260709 13:03:15.110198 15232 rpcz_store.cc:276] 0709 13:03:13.801648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.981527 (+1179879us) service_pool.cc:224] Handling call
0709 13:03:15.077118 (+ 95591us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.031150 15843 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.034s	sys 0.080s
I20260709 13:03:15.076310 15840 rpcz_store.cc:276] 0709 13:03:14.063119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.048239 (+985120us) service_pool.cc:224] Handling call
0709 13:03:15.075812 (+ 27573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.093247 17298 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.076004 15841 rpcz_store.cc:276] 0709 13:03:13.615788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.620853 (+  5065us) service_pool.cc:224] Handling call
0709 13:03:13.663586 (+ 42733us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.708775 (+ 45189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.710058 (+  1283us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.749486 (+ 39428us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.889555 (+140069us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:13.893582 (+  4027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.978012 (+ 84430us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.822190 (+844178us) raft_consensus.cc:1685] finished
0709 13:03:14.822209 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.970564 (+148355us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24293,"raft.queue_time_us":12027,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7163,"thread_start_us":76506,"threads_started":1,"wal-append.queue_time_us":316066,"wal-append.run_cpu_time_us":2188,"wal-append.run_wall_time_us":1002200}
I20260709 13:03:15.095469 15847 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:15.077435 17355 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.109686 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 427) took 1578 ms. Trace:
I20260709 13:03:15.089599 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 413) took 6221 ms. Trace:
I20260709 13:03:15.067008 17268 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:15.109670 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.113440 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 417) took 5790 ms. Trace:
I20260709 13:03:15.115788 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 718) took 1053 ms. Trace:
I20260709 13:03:15.108539 17222 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:15.129688 15844 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:15.129009 15847 rpcz_store.cc:276] 0709 13:03:14.062069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.026695 (+964626us) service_pool.cc:224] Handling call
0709 13:03:15.115719 (+ 89024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.128772 15228 rpcz_store.cc:276] 0709 13:03:09.322993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.080907 (+5757914us) service_pool.cc:224] Handling call
0709 13:03:15.112774 (+ 31867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.113037 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.113065 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.113109 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.113204 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.113263 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.113382 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.131068 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.131655 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 701) took 1168 ms. Trace:
I20260709 13:03:15.131567 15232 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.132099 15847 rpcz_store.cc:276] 0709 13:03:13.963360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.130815 (+1167455us) service_pool.cc:224] Handling call
0709 13:03:15.131610 (+   795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.132670 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 418) took 5788 ms. Trace:
I20260709 13:03:15.050066 17304 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.133337 15232 rpcz_store.cc:276] 0709 13:03:09.343986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.130903 (+5786917us) service_pool.cc:224] Handling call
0709 13:03:15.131316 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.132335 (+  1019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.132593 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.133248 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.038049 17106 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:15.135219 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 702) took 1171 ms. Trace:
W20260709 13:03:15.113646 15843 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.036s	sys 0.081s
I20260709 13:03:15.135914 15847 rpcz_store.cc:276] 0709 13:03:13.963610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.132695 (+1169085us) service_pool.cc:224] Handling call
0709 13:03:15.135149 (+  2454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.137701 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.134085 17208 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.077865 17364 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.136674 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.115341 17298 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.080375 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2210) took 1331 ms. Trace:
I20260709 13:03:15.077330 15088 rpcz_store.cc:276] 0709 13:03:11.975357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.975508 (+   151us) service_pool.cc:224] Handling call
0709 13:03:11.975809 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.177902 (+202093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.052695 (+2874793us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003250d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.053149 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361179008}
I20260709 13:03:15.163755 17387 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.128050 15236 rpcz_store.cc:276] 0709 13:03:08.867670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.042600 (+6174930us) service_pool.cc:224] Handling call
0709 13:03:15.088996 (+ 46396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.089209 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.089236 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.089287 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.089330 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.089418 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.089534 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.253155 17387 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.255134 15236 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:15.256132 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1710) took 4281 ms. Trace:
I20260709 13:03:15.175731 17364 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.176196 17338 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.259243 17387 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.142115 17326 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.157989 17389 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.262252 17326 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.263640 17389 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.166653 17390 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.266114 17390 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.219237 15079 rpcz_store.cc:276] 0709 13:03:13.746266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.845736 (+ 99470us) service_pool.cc:224] Handling call
0709 13:03:15.077173 (+1231437us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.269536 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 447630us
I20260709 13:03:15.272091 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 720) took 1209 ms. Trace:
I20260709 13:03:15.272714 15840 rpcz_store.cc:276] 0709 13:03:14.062619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.115308 (+1052689us) service_pool.cc:224] Handling call
0709 13:03:15.271692 (+156384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.138818 15847 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:15.273607 17387 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.274462 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 716) took 1212 ms. Trace:
I20260709 13:03:15.274679 17387 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.275090 15847 rpcz_store.cc:276] 0709 13:03:14.061487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.137510 (+1076023us) service_pool.cc:224] Handling call
0709 13:03:15.274203 (+136693us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.111164 15233 rpcz_store.cc:269] 0709 13:03:12.973280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.793486 (+820206us) service_pool.cc:224] Handling call
0709 13:03:15.085182 (+1291696us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.277477 15129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.044s	sys 0.063s
I20260709 13:03:15.277872 15233 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:15.278074 17387 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:15.278157 15129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.044s	sys 0.063s
I20260709 13:03:15.279959 17132 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:15.280416 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.281304 17310 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.281253 17132 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.282303 15841 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:15.283041 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 724) took 1219 ms. Trace:
I20260709 13:03:15.282790 17310 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.274891 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.283545 15841 rpcz_store.cc:276] 0709 13:03:14.063638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.143695 (+1080057us) service_pool.cc:224] Handling call
0709 13:03:15.282920 (+139225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.299218 15840 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:15.300349 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 706) took 1335 ms. Trace:
I20260709 13:03:15.300516 17310 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.256539 15236 rpcz_store.cc:276] 0709 13:03:10.974550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.253644 (+4279094us) service_pool.cc:224] Handling call
0709 13:03:15.254620 (+   976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.255678 (+  1058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.255993 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.140957 17208 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.300935 15840 rpcz_store.cc:276] 0709 13:03:13.964920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.274223 (+1309303us) service_pool.cc:224] Handling call
0709 13:03:15.300269 (+ 26046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.304314 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.163429 17309 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:15.312650 17309 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:15.153041 17139 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:15.314340 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1711) took 4339 ms. Trace:
I20260709 13:03:15.183422 17355 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.314782 15233 rpcz_store.cc:276] 0709 13:03:10.974829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.276830 (+4302001us) service_pool.cc:224] Handling call
0709 13:03:15.277405 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.293289 (+ 15884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.314232 (+ 20943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.153105 17222 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.307355 17338 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.140059 17268 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.319272 17338 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.319711 17268 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.322314 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 757) took 1688 ms. Trace:
I20260709 13:03:15.310088 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 587) took 3335 ms. Trace:
I20260709 13:03:15.311460 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:15.313776 15086 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 601) took 2803 ms (client timeout 1770 ms). Trace:
I20260709 13:03:15.342387 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.311457 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 592) took 3288 ms. Trace:
W20260709 13:03:15.368446 15086 rpcz_store.cc:269] 0709 13:03:12.510553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.528857 (+ 18304us) service_pool.cc:224] Handling call
0709 13:03:15.313277 (+2784420us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540025ba54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.313514 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395130752}
I20260709 13:03:15.323019 15842 rpcz_store.cc:276] 0709 13:03:13.633481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.892722 (+259241us) service_pool.cc:224] Handling call
0709 13:03:13.955267 (+ 62545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.017807 (+ 62540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.019448 (+  1641us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.089089 (+ 69641us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.113262 (+1024173us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.114309 (+  1047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.134703 (+ 20394us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.321733 (+187030us) raft_consensus.cc:1685] finished
0709 13:03:15.321782 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.322080 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10740,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2856,"thread_start_us":990963,"threads_started":1,"wal-append.queue_time_us":1018020}
I20260709 13:03:15.138329 15232 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 13:03:15.393770 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1709) took 4419 ms. Trace:
I20260709 13:03:15.116181 15163 rpcz_store.cc:276] 0709 13:03:13.531769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.108810 (+1577041us) service_pool.cc:224] Handling call
0709 13:03:15.109152 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.109333 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.109364 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.109405 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.109448 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.109500 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.109634 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.395102 17220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.012s	sys 0.008s
I20260709 13:03:15.396518 17208 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.397275 17222 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.397485 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 704) took 1433 ms. Trace:
I20260709 13:03:15.397606 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1700) took 5320 ms. Trace:
I20260709 13:03:15.385991 15081 rpcz_store.cc:276] 0709 13:03:12.022595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.079115 (+ 56520us) service_pool.cc:224] Handling call
0709 13:03:12.079488 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.971830 (+892342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.310900 (+2339070us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54001b12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.311294 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2050288512}
I20260709 13:03:15.398552 15232 rpcz_store.cc:276] 0709 13:03:10.974271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.137364 (+4163093us) service_pool.cc:224] Handling call
0709 13:03:15.137763 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.392933 (+255170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.393509 (+   576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.366695 17390 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.335865 15080 rpcz_store.cc:276] 0709 13:03:11.974229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.974416 (+   187us) service_pool.cc:224] Handling call
0709 13:03:11.974820 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.329701 (+354881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.309502 (+2979801us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400118854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.309870 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789797120}
I20260709 13:03:15.401417 15228 rpcz_store.cc:276] 0709 13:03:10.077375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.136156 (+5058781us) service_pool.cc:224] Handling call
0709 13:03:15.305526 (+169370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.306125 (+   599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.306153 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.306250 (+    97us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:15.397091 (+ 90841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.397183 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.397332 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.408726 17220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.124s	user 0.013s	sys 0.009s
I20260709 13:03:15.409719 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 717) took 1348 ms. Trace:
I20260709 13:03:15.410534 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.411736 15841 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:15.410293 15847 rpcz_store.cc:276] 0709 13:03:14.061779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.276343 (+1214564us) service_pool.cc:224] Handling call
0709 13:03:15.409635 (+133292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.414783 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 599) took 3052 ms. Trace:
W20260709 13:03:15.414896 15128 connection.cc:256] RPC call timeout handler was delayed by -1.5711566219999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 13:03:15.415550 15128 connection.cc:256] RPC call timeout handler was delayed by -1.5264263240001128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:03:15.149371 17331 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.168493 17304 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.173715 17329 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.418658 17309 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:15.419727 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1612) took 3823 ms (client timeout 1650 ms). Trace:
I20260709 13:03:15.421944 17355 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.422156 15232 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
W20260709 13:03:15.422745 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1723) took 4172 ms (client timeout 2410 ms). Trace:
I20260709 13:03:14.961772 15839 rpcz_store.cc:276] 0709 13:03:11.504119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.812826 (+308707us) service_pool.cc:224] Handling call
0709 13:03:11.813345 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.107781 (+294436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.116452 (+  8671us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.185372 (+ 68920us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.548990 (+363618us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:12.570080 (+ 21090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.954563 (+1384483us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400806454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:13.954625 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.954683 (+    58us) raft_consensus.cc:1685] finished
0709 13:03:13.954703 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.961508 (+  6805us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25366,"raft.queue_time_us":26668,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7264,"spinlock_wait_cycles":602591616,"thread_start_us":307339,"threads_started":1,"wal-append.queue_time_us":482792,"wal-append.run_cpu_time_us":14919,"wal-append.run_wall_time_us":1866538}
I20260709 13:03:15.423648 17132 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.424661 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.425706 17106 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.425969 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 428) took 1755 ms. Trace:
I20260709 13:03:15.426355 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:15.426994 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.06279e+06us
I20260709 13:03:15.426528 15163 rpcz_store.cc:276] 0709 13:03:13.670862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.424171 (+1753309us) service_pool.cc:224] Handling call
0709 13:03:15.424898 (+   727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.425373 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.425410 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.425483 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.425531 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.425597 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.425729 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.427349 15839 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:15.428637 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:15.429736 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.13143e+06us
I20260709 13:03:15.430426 17389 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.431389 17389 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:15.431774 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 416) took 6114 ms. Trace:
I20260709 13:03:15.150802 17383 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.433490 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.434370 17383 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.435130 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 756) took 1802 ms. Trace:
I20260709 13:03:15.435891 15847 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:15.402635 15840 rpcz_store.cc:276] 0709 13:03:13.964375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.303719 (+1339344us) service_pool.cc:224] Handling call
0709 13:03:15.397368 (+ 93649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.433511 17268 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.423141 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1717) took 4346 ms. Trace:
I20260709 13:03:15.441951 15232 rpcz_store.cc:276] 0709 13:03:11.077082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.421132 (+4344050us) service_pool.cc:224] Handling call
0709 13:03:15.421773 (+   641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.422793 (+  1020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.423061 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.444056 15232 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:15.446079 17053 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.447117 17053 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.436753 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 721) took 1373 ms. Trace:
I20260709 13:03:15.449386 15847 rpcz_store.cc:276] 0709 13:03:14.062858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.434819 (+1371961us) service_pool.cc:224] Handling call
0709 13:03:15.435491 (+   672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.436388 (+   897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.436647 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.444749 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1718) took 4367 ms. Trace:
I20260709 13:03:15.457760 17139 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.457324 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 630200us
I20260709 13:03:15.460999 15232 rpcz_store.cc:276] 0709 13:03:11.077541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.443040 (+4365499us) service_pool.cc:224] Handling call
0709 13:03:15.443665 (+   625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.444515 (+   850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.444697 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.432387 15234 rpcz_store.cc:276] 0709 13:03:09.317576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.078713 (+5761137us) service_pool.cc:224] Handling call
0709 13:03:15.084184 (+  5471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.084483 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.084523 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.084587 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.084653 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.084734 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.431299 (+346565us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b5400295750. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.431546 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11543424}
I20260709 13:03:15.440583 14984 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 260) took 9749 ms. Trace:
W20260709 13:03:15.418659 15229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1742) took 3448 ms (client timeout 2136 ms). Trace:
W20260709 13:03:15.423278 15235 rpcz_store.cc:269] 0709 13:03:11.250051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.422249 (+4172198us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:15.422509 (+   260us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:15.448040 17053 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:15.420347 15231 rpcz_store.cc:269] 0709 13:03:11.596367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.034955 (+438588us) service_pool.cc:224] Handling call
0709 13:03:15.419211 (+3384256us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54002783d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.419462 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654862208}
I20260709 13:03:15.435709 15848 rpcz_store.cc:276] 0709 13:03:13.632774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.880830 (+248056us) service_pool.cc:224] Handling call
0709 13:03:13.902510 (+ 21680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.074463 (+1171953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.434572 (+360109us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54006d4054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.434914 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2694002944}
I20260709 13:03:14.858753 15156 rpcz_store.cc:276] 0709 13:03:11.976818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.111538 (+134720us) service_pool.cc:224] Handling call
0709 13:03:12.112042 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.294840 (+182798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.297987 (+  3147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.303111 (+  5124us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:12.304746 (+  1635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.894970 (+1590224us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400287dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:13.895041 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.034723 (+139682us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113709,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":13511,"spinlock_wait_cycles":54792064,"thread_start_us":126774,"threads_started":1}
I20260709 13:03:15.468928 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 705) took 1499 ms. Trace:
I20260709 13:03:15.469445 15841 rpcz_store.cc:276] 0709 13:03:13.964650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.404723 (+1440073us) service_pool.cc:224] Handling call
0709 13:03:15.463901 (+ 59178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.472890 15237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1743) took 2472 ms (client timeout 2171 ms). Trace:
I20260709 13:03:15.473325 17326 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.468964 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1727) took 4214 ms. Trace:
I20260709 13:03:15.415444 15083 rpcz_store.cc:276] 0709 13:03:12.362544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.651866 (+289322us) service_pool.cc:224] Handling call
0709 13:03:12.660167 (+  8301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.152239 (+1492072us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b1800283540. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9dcbfb 00007f0d3f9e764b
0709 13:03:14.152452 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.152491 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.859099 (+706608us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.392818 (+533719us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.392971 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.393146 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.411658 (+ 18512us) raft_consensus.cc:1685] finished
0709 13:03:15.411684 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.414558 (+  2874us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26289,"raft.queue_time_us":233103,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":14274,"spinlock_wait_cycles":138606848,"thread_start_us":665363,"threads_started":2,"wal-append.queue_time_us":440811}
I20260709 13:03:15.420666 17309 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:15.463634 15229 rpcz_store.cc:269] 0709 13:03:11.970044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.669140 (+699096us) service_pool.cc:224] Handling call
0709 13:03:15.418126 (+2748986us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54000582d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.418344 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1987961728}
I20260709 13:03:15.483939 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.484961 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 698) took 1834 ms. Trace:
I20260709 13:03:15.485153 15229 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.485692 17178 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:15.486228 17178 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.486621 17178 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:15.487073 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.488137 15229 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:15.481650 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 170691us
I20260709 13:03:15.489693 15228 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:15.490490 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1714) took 4514 ms. Trace:
I20260709 13:03:15.490849 15228 rpcz_store.cc:276] 0709 13:03:10.975601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.463505 (+4487904us) service_pool.cc:224] Handling call
0709 13:03:15.489309 (+ 25804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.490168 (+   859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.490399 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.491866 17285 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.492992 17220 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:15.494133 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.493937 17220 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.496480 17220 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.491950 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.497263 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.499032 17383 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.492925 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.501088 15231 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:15.501711 17306 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 3.004s	user 0.019s	sys 0.031s
I20260709 13:03:15.502319 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.502560 15231 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.503036 15156 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.503255 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 421) took 3267 ms. Trace:
I20260709 13:03:15.503609 15231 rpcz_store.cc:276] 0709 13:03:12.235738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.501863 (+3266125us) service_pool.cc:224] Handling call
0709 13:03:15.502326 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.503026 (+   700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.503200 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.504087 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.496628 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:15.473371 15237 rpcz_store.cc:269] 0709 13:03:12.972828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.731066 (+758238us) service_pool.cc:224] Handling call
0709 13:03:15.444485 (+1713419us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b54000591d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.444682 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209411456}
I20260709 13:03:15.505298 15231 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:15.505898 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1616) took 3018 ms. Trace:
I20260709 13:03:15.506153 15237 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:15.506229 15231 rpcz_store.cc:276] 0709 13:03:12.487808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.504374 (+3016566us) service_pool.cc:224] Handling call
0709 13:03:15.504775 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.505645 (+   870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.505832 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.506942 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1617) took 3018 ms. Trace:
I20260709 13:03:15.507309 15237 rpcz_store.cc:276] 0709 13:03:12.488031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.505468 (+3017437us) service_pool.cc:224] Handling call
0709 13:03:15.505804 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.506726 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.506893 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.507725 15231 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:15.509104 17364 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.509493 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1618) took 3020 ms. Trace:
I20260709 13:03:15.509877 17364 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.510062 15231 rpcz_store.cc:276] 0709 13:03:12.488253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.507073 (+3018820us) service_pool.cc:224] Handling call
0709 13:03:15.507418 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.508085 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.508243 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.476271 15234 rpcz_store.cc:276] 0709 13:03:11.254172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.467117 (+4212945us) service_pool.cc:224] Handling call
0709 13:03:15.467694 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.468422 (+   728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.468466 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.468553 (+    87us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:15.468604 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.468690 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.468890 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4712,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":236}
I20260709 13:03:15.513415 15228 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.468721 17053 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.514637 15237 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:15.517310 15839 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.518129 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 707) took 1552 ms. Trace:
I20260709 13:03:15.518864 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:15.520186 17218 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:15.520679 17218 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.521174 17218 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.523017 17391 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:15.523566 17391 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.523746 17348 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.523960 17391 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.524308 17364 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:15.525388 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.525682 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:15.515275 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.527743 15235 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.528647 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1621) took 3026 ms. Trace:
I20260709 13:03:15.530637 15231 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:15.530933 15237 rpcz_store.cc:276] 0709 13:03:12.490087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.508041 (+3017954us) service_pool.cc:224] Handling call
0709 13:03:15.508497 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.515063 (+  6566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.515242 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.518543 15839 rpcz_store.cc:276] 0709 13:03:13.965210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.428469 (+1463259us) service_pool.cc:224] Handling call
0709 13:03:15.516940 (+ 88471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.517765 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.518039 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.536085 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.539073 15082 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.541054 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:15.542361 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.541637 15235 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.543151 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.543752 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.544620 15235 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:15.545094 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 690) took 3226 ms (client timeout 1762 ms). Trace:
I20260709 13:03:15.541304 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.545827 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.541919 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.550786 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:15.485496 15843 rpcz_store.cc:276] 0709 13:03:13.650307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.954666 (+304359us) service_pool.cc:224] Handling call
0709 13:03:13.961073 (+  6407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.961571 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:15.142256 (+1180685us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.484215 (+341959us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.484326 (+   111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.484443 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.484566 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.484731 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":76292,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":18501,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"thread_start_us":1351417,"threads_started":2,"wal-append.queue_time_us":309330}
I20260709 13:03:15.551195 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.552889 17383 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:15.553669 17383 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.554177 17383 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.546319 17139 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.552953 15156 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:15.545544 15846 rpcz_store.cc:269] 0709 13:03:12.318253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.318410 (+   157us) service_pool.cc:224] Handling call
0709 13:03:15.544626 (+3226216us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54007e4354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.544873 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488977280}
I20260709 13:03:15.564735 17277 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:15.565261 17355 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.565707 17277 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.566200 17355 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.567751 17297 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.568797 17318 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:15.569036 17132 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.569727 17318 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.569862 17132 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.572436 17318 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.573436 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:15.574635 15154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 423) took 3080 ms (client timeout 1770 ms). Trace:
I20260709 13:03:15.574731 17398 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:15.575618 17398 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:15.575480 15154 rpcz_store.cc:269] 0709 13:03:12.493942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.543213 (+ 49271us) service_pool.cc:224] Handling call
0709 13:03:15.574102 (+3030889us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540013ecd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.574318 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373896064}
I20260709 13:03:15.576133 17398 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:15.576824 15846 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:15.579728 17298 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.581990 15232 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:15.582661 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1726) took 4328 ms. Trace:
I20260709 13:03:15.583019 15232 rpcz_store.cc:276] 0709 13:03:11.253908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.462999 (+4209091us) service_pool.cc:224] Handling call
0709 13:03:15.537948 (+ 74949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.582396 (+ 44448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.582607 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.583679 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.584033 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:15.584496 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.585374 17326 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.586472 15155 consensus_meta.cc:287] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.009s
I20260709 13:03:15.586148 17326 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.586937 15155 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
W20260709 13:03:15.587530 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 436) took 1708 ms (client timeout 1985 ms). Trace:
W20260709 13:03:15.587879 15155 rpcz_store.cc:269] 0709 13:03:13.878981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.989317 (+1110336us) service_pool.cc:224] Handling call
0709 13:03:15.587430 (+598113us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.588483 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.589157 17400 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.590701 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2321) took 3220 ms. Trace:
I20260709 13:03:15.594643 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:15.603982 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45684.5us
I20260709 13:03:15.586486 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.590472 14982 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.612527 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 755) took 1979 ms. Trace:
I20260709 13:03:15.613214 15844 rpcz_store.cc:276] 0709 13:03:13.632535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.871784 (+239249us) service_pool.cc:224] Handling call
0709 13:03:13.872121 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.130319 (+1258198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.611893 (+481574us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400852ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.612324 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2891547008}
I20260709 13:03:15.615417 17178 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.616149 17132 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.612648 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:15.619120 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 708) took 2907 ms (client timeout 1931 ms). Trace:
I20260709 13:03:15.619416 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.620229 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:15.584582 15233 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.620522 15840 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:15.621062 17106 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.683583 15843 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:15.683495 17106 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.684880 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.685647 16998 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:15.609560 17355 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.602365 15160 rpcz_store.cc:276] 0709 13:03:12.370294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.838601 (+468307us) service_pool.cc:224] Handling call
0709 13:03:12.838881 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.561678 (+722797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.590103 (+2028425us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.590419 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475712768}
I20260709 13:03:15.694064 16998 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.695036 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.695928 15160 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:15.624151 16946 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.592105 15232 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.621579 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:15.650234 15848 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:15.675705 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1746) took 2701 ms. Trace:
I20260709 13:03:15.588578 17326 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.699743 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:15.679890 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.677640 15845 rpcz_store.cc:269] 0709 13:03:12.711360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.839806 (+128446us) service_pool.cc:224] Handling call
0709 13:03:15.618651 (+2778845us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54004b55d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.618883 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150324992}
I20260709 13:03:15.685071 17214 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.621548 17318 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:15.683611 15054 connection.cc:256] RPC call timeout handler was delayed by -1.4873742450000691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:03:15.698575 15847 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:15.702251 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 253888us
I20260709 13:03:15.698045 15234 rpcz_store.cc:276] 0709 13:03:12.973927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.528904 (+2554977us) service_pool.cc:224] Handling call
0709 13:03:15.581303 (+ 52399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.638635 (+ 57332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.638684 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.675291 (+ 36607us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:15.675352 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.675426 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.675615 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27089,"raft.queue_time_us":70394,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156}
I20260709 13:03:15.650234 17400 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.715427 17302 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.719190 17139 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.721271 15083 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:03:15.716159 17302 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.722224 17383 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.723847 17302 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.724722 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:15.725021 15810 leader_election.cc:336] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.718s (SENT)
I20260709 13:03:15.725904 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:15.726363 15839 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.729354 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.729702 17214 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.730579 15079 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.733037 15088 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:15.734747 17277 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.736706 17299 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:15.737545 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 161642us
I20260709 13:03:15.737928 16946 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:15.738713 17353 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:15.739362 17353 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:15.739557 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:15.740985 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.741436 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.743196 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:15.743865 15155 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:15.728454 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 190062us
I20260709 13:03:15.726137 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:15.744997 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.745565 15086 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:15.737285 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 429) took 2056 ms. Trace:
I20260709 13:03:15.747290 15154 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:15.746554 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.740598 15848 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:15.725208 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.708616 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.706532 15236 raft_consensus.cc:2376] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2. Re-sending same reply.
I20260709 13:03:15.716511 17400 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.748411 15081 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:15.757359 15203 connection.cc:256] RPC call timeout handler was delayed by -1.3149152080000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:03:15.761888 15203 connection.cc:256] RPC call timeout handler was delayed by -1.3042782140000782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:03:15.747083 15163 rpcz_store.cc:276] 0709 13:03:13.680945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.428214 (+1747269us) service_pool.cc:224] Handling call
0709 13:03:15.451323 (+ 23109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.458453 (+  7130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.458494 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.459358 (+   864us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.515453 (+ 56095us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.515498 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.515558 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.736949 (+221391us) raft_consensus.cc:1685] finished
0709 13:03:15.736978 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.737179 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5805,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":286,"thread_start_us":54150,"threads_started":1,"wal-append.queue_time_us":150538}
W20260709 13:03:15.769663 15051 leader_election.cc:336] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.326s (SENT)
I20260709 13:03:15.720300 17398 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.749742 17106 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.782200 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:15.776474 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.755360 16998 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.754630 17383 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.784258 15234 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.747968 17204 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.782217 17400 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.792227 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:15.770201 17214 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.794222 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.802560 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.797663 17400 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.805122 17404 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:03:15.809481 15848 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:15.810940 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.811717 15234 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.792928 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.819576 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.817512 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:15.822999 15810 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.749s (SENT)
I20260709 13:03:15.823559 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:15.824601 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.825946 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.802688 17208 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.828425 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.826691 15234 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.778411 17218 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:15.827361 15126 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.954s (SENT)
I20260709 13:03:15.810396 16998 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:15.464124 14984 rpcz_store.cc:276] 0709 13:03:05.690943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.746827 (+ 55884us) service_pool.cc:224] Handling call
0709 13:03:06.167030 (+420203us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:15.440308 (+9273278us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.914391 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.915027 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.915077 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:03:11.915095 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:11.915778 (+   683us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.915841 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.915872 (+    31us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:03:11.916289 (+   417us) tablet.cc:701] Row locks acquired
0709 13:03:11.916306 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:03:11.916494 (+   188us) write_op.cc:270] Start()
0709 13:03:11.916625 (+   131us) write_op.cc:276] Timestamp: P: 1783602191916434 usec, L: 0
0709 13:03:11.916650 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:03:11.931804 (+ 15154us) log.cc:844] Serialized 1311 byte log entry
0709 13:03:12.084462 (+152658us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3c8da0ac 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b
0709 13:03:12.677321 (+592859us) op_driver.cc:464] REPLICATION: finished
0709 13:03:13.519710 (+842389us) write_op.cc:301] APPLY: starting
0709 13:03:13.523104 (+  3394us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:13.544740 (+ 21636us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:13.548081 (+  3341us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:13.736740 (+188659us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:13.736996 (+   256us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:03:13.737044 (+    48us) write_op.cc:312] APPLY: finished
0709 13:03:13.737622 (+   578us) log.cc:844] Serialized 35 byte log entry
0709 13:03:13.886868 (+149246us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.898765 (+ 11897us) write_op.cc:454] Released schema lock
0709 13:03:13.900072 (+  1307us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:14.187061 (+286989us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:14.195743 (+  8682us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":492426,"reported_tablets":6,"spinlock_wait_cycles":325662464,"child_traces":[["op",{"apply.queue_time_us":757642,"apply.run_cpu_time_us":28721,"apply.run_wall_time_us":576035,"num_ops":3,"prepare.queue_time_us":521784,"prepare.run_cpu_time_us":16247,"prepare.run_wall_time_us":242661,"raft.queue_time_us":294604,"raft.run_cpu_time_us":3065,"raft.run_wall_time_us":47790,"replication_time_us":732728,"spinlock_wait_cycles":13911296,"thread_start_us":800186,"threads_started":5,"wal-append.queue_time_us":590759,"wal-append.run_cpu_time_us":24797,"wal-append.run_wall_time_us":2300780}]]}
I20260709 13:03:15.800724 17106 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:15.836391 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:15.837695 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 80866.6us
I20260709 13:03:15.840008 15163 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.840745 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.842252 15156 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:15.842669 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.781387 17353 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.858620 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.846222 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.849797 15085 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:15.860639 15087 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:15.859390 15234 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.858408 15236 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:15.862746 15086 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:15.865167 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:15.865587 15237 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:15.866585 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43630.6us
I20260709 13:03:15.865484 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.868752 17398 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:15.871662 17106 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:15.873750 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.874584 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.875010 17106 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.877828 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.932478 15229 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:15.886253 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:15.933408 15230 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.888818 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.890465 17338 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.860275 17400 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:15.894574 15088 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:15.886838 15160 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:15.861191 15233 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.881856 17200 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:03:15.925575 15810 leader_election.cc:336] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.764s (SENT)
I20260709 13:03:15.839661 17390 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.839434 17298 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:15.926795 15083 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:15.948736 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 126234us
I20260709 13:03:15.878242 15841 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:15.885442 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.933601 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:15.934057 15237 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:15.889679 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 723) took 1826 ms. Trace:
I20260709 13:03:15.959923 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:15.935565 15848 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:15.934943 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:15.881132 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:15.961771 17400 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:15.962078 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 155475us
I20260709 13:03:15.962589 15235 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.963064 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:15.964072 17400 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:15.964465 17400 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.964774 17400 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:15.965476 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.964293 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:15.966384 15156 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:15.966572 17222 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:15.967777 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:15.968166 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:15.972014 15087 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:15.973809 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.936432 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1748) took 2961 ms. Trace:
I20260709 13:03:15.996981 15229 rpcz_store.cc:276] 0709 13:03:12.974580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.530373 (+2555793us) service_pool.cc:224] Handling call
0709 13:03:15.888057 (+357684us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:15.924478 (+ 36421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.936045 (+ 11567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.936347 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71528704}
I20260709 13:03:15.941344 15232 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:15.868623 17293 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.945901 15163 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:15.939725 17398 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:15.942780 15814 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.985s (SENT) [suppressed 49 similar messages]
I20260709 13:03:15.844345 17208 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:15.903594 17391 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.944283 17353 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:15.945705 15051 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.524s (SENT)
I20260709 13:03:15.973299 17400 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.978871 15154 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:15.968540 15156 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:15.979880 17338 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:15.981194 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55657.8us
W20260709 13:03:16.029109 15810 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.953s (SENT)
I20260709 13:03:16.029475 17326 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.029570 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:16.030004 17326 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:16.031014 17326 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:16.032184 17285 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:16.032359 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:16.025767 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:15.982015 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 418) took 3494 ms (client timeout 1749 ms). Trace:
I20260709 13:03:16.033725 16946 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:16.034468 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1605) took 2338 ms. Trace:
I20260709 13:03:16.034891 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.036857 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:15.987494 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 430) took 2285 ms (client timeout 2221 ms). Trace:
W20260709 13:03:16.038681 15158 rpcz_store.cc:269] 0709 13:03:13.701896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.770119 (+ 68223us) service_pool.cc:224] Handling call
0709 13:03:15.987102 (+2216983us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b540017e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.987273 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27583744}
I20260709 13:03:16.039098 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:16.039865 15842 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:16.040132 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:15.990233 15839 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:16.040534 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:16.040937 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:16.041610 15842 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.958146 15840 rpcz_store.cc:276] 0709 13:03:14.063380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.438114 (+1374734us) service_pool.cc:224] Handling call
0709 13:03:15.619826 (+181712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.682769 (+ 62943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.889347 (+206578us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54004b55d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.889562 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416664448}
I20260709 13:03:15.991973 17390 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:15.993393 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:16.043215 15237 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:16.035588 15234 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:16.044884 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1749) took 3070 ms. Trace:
I20260709 13:03:16.045285 15234 rpcz_store.cc:276] 0709 13:03:12.974882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.959638 (+2984756us) service_pool.cc:224] Handling call
0709 13:03:16.035245 (+ 75607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.044598 (+  9353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.044811 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.017413 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.034860 16946 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.051069 15230 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:15.946561 17200 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:16.000134 17233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.181s	user 0.029s	sys 0.065s
I20260709 13:03:16.064004 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:16.002799 15813 connection.cc:256] RPC call timeout handler was delayed by -1.8006685760000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 13:03:16.002895 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:16.066759 15160 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:16.018006 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:16.020460 17117 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.020960 17218 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:16.021531 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 434) took 2153 ms. Trace:
I20260709 13:03:16.073951 15156 rpcz_store.cc:276] 0709 13:03:13.867673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.020188 (+2152515us) service_pool.cc:224] Handling call
0709 13:03:16.020643 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.021104 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.021138 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.021191 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:16.021248 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.021326 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.021454 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2635,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":235}
I20260709 13:03:16.022329 17208 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:16.077419 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.023200 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 432) took 2310 ms. Trace:
I20260709 13:03:16.078246 15158 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:16.078085 15154 rpcz_store.cc:276] 0709 13:03:13.713121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.022166 (+2309045us) service_pool.cc:224] Handling call
0709 13:03:16.022616 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.022848 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.022874 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.022916 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:16.022958 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.023017 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.023117 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.079752 15158 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:16.024206 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:16.071413 17317 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:03:16.024340 17391 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:16.025202 15228 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.024945 17293 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.025843 15844 raft_consensus.cc:2368] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:16.025924 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.083583 17314 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:03:16.027741 17400 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:15.982167 15162 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:16.086906 15237 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:16.040709 17178 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:03:15.986267 17404 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:16.089952 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.090891 15237 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:16.033742 15157 rpcz_store.cc:269] 0709 13:03:12.487857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.499735 (+ 11878us) service_pool.cc:224] Handling call
0709 13:03:15.981563 (+3481828us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b54001c6bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:15.981772 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886819200}
W20260709 13:03:16.093464 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 147911us
I20260709 13:03:16.049928 16946 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:16.048043 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:16.000373 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.003755 15086 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:16.067958 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:16.072010 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.084775 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.095518 15234 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:16.072366 15163 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.050130 15233 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:16.073158 17326 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:16.083107 17411 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:03:16.065912 17233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.445s	user 0.029s	sys 0.065s
I20260709 13:03:16.034858 15082 rpcz_store.cc:276] 0709 13:03:13.696431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.734685 (+ 38254us) service_pool.cc:224] Handling call
0709 13:03:13.751526 (+ 16841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.550612 (+1799086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.034156 (+483544us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b5400191c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:16.034390 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4086099712}
I20260709 13:03:15.945174 17106 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.052438 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.103652 15230 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:15.942701 15155 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:16.371802 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.266358 15158 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:16.281898 15160 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:16.239467 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.376382 17233 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.136384 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.227293 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:16.120877 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.258076 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 288230us
W20260709 13:03:16.380467 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.378357 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 410243us
I20260709 13:03:16.319730 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1624) took 3792 ms. Trace:
W20260709 13:03:16.372702 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 324719us
W20260709 13:03:16.253077 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 220776us
I20260709 13:03:16.302446 16946 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.383770 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.099363 17218 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:16.377671 15842 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:16.377224 17233 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.383190 15231 rpcz_store.cc:276] 0709 13:03:12.494454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.511077 (+3016623us) service_pool.cc:224] Handling call
0709 13:03:15.514439 (+  3362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.531151 (+ 16712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.286289 (+755138us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b54003d95d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:16.286502 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6950272}
I20260709 13:03:16.386250 15231 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.386572 17233 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:16.377923 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:16.387904 15236 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:16.378510 15085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2212) took 2631 ms (client timeout 1647 ms). Trace:
I20260709 13:03:16.154657 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.115231 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2324) took 3728 ms. Trace:
I20260709 13:03:16.376519 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.399418 15234 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:16.399813 15232 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:16.179915 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1750) took 3154 ms. Trace:
I20260709 13:03:16.085049 17317 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.378065 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:16.381920 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 538206us
I20260709 13:03:16.401467 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:16.404080 15235 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:16.404095 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.404641 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.399120 15085 rpcz_store.cc:269] 0709 13:03:13.746590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.753024 (+  6434us) service_pool.cc:224] Handling call
0709 13:03:16.176238 (+2423214us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54002373d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:16.378433 (+202195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4656611584}
W20260709 13:03:16.405439 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.213903 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:16.386993 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1754) took 3360 ms. Trace:
I20260709 13:03:16.376555 15160 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:03:16.377305 15813 connection.cc:256] RPC call timeout handler was delayed by -1.0937262920000421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:03:16.406759 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:16.407099 15813 connection.cc:256] RPC call timeout handler was delayed by -1.0181262920000336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:03:16.406718 15231 rpcz_store.cc:276] 0709 13:03:13.026756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.385562 (+3358806us) service_pool.cc:224] Handling call
0709 13:03:16.385918 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.386725 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.386935 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:16.372444 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 432133us
I20260709 13:03:16.153908 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:16.400384 15228 rpcz_store.cc:276] 0709 13:03:13.025453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.977940 (+2952487us) service_pool.cc:224] Handling call
0709 13:03:15.978383 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.082756 (+104373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.179567 (+ 96811us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b5400057dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:16.179823 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106854272}
I20260709 13:03:16.401078 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:16.408529 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.399613 15159 rpcz_store.cc:276] 0709 13:03:12.374443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.434043 (+1059600us) service_pool.cc:224] Handling call
0709 13:03:13.475613 (+ 41570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.714020 (+238407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.102475 (+2388455us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54003e1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:16.102839 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432141696}
I20260709 13:03:16.396762 15163 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:16.409816 15230 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:16.100366 17309 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:16.384513 17218 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.408003 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:16.408996 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 475212us
W20260709 13:03:16.415519 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.406700 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.414865 16946 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:16.154134 17299 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:16.420463 15228 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:16.420640 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.100261 17222 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:16.422981 15232 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:16.426824 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:16.298151 17390 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:16.405676 17053 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:16.424584 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.417975 17218 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:16.407713 17317 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:16.135062 17106 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.456357 15234 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.457567 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1756) took 3430 ms. Trace:
I20260709 13:03:16.458751 15234 rpcz_store.cc:276] 0709 13:03:13.027348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.419977 (+3392629us) service_pool.cc:224] Handling call
0709 13:03:16.450316 (+ 30339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.457311 (+  6995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.457508 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.446498 15127 leader_election.cc:400] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:16.447793 15086 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
W20260709 13:03:16.448045 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63817.1us
I20260709 13:03:16.448374 15080 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.439147 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.460011 17313 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:16.460086 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 306322us
I20260709 13:03:16.460968 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:16.461529 15230 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:03:16.462793 17379 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.323s	user 0.003s	sys 0.002s
I20260709 13:03:16.468322 17391 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:16.465507 17317 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.466270 15232 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:16.452226 15082 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:16.483271 15127 leader_election.cc:403] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:16.435894 17404 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:16.445561 15236 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:16.584883 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 428) took 1168 ms. Trace:
I20260709 13:03:16.451709 17297 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:16.439929 17053 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.463959 17293 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:16.449343 17398 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:16.585618 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 146297us
I20260709 13:03:16.585295 15236 rpcz_store.cc:276] 0709 13:03:15.416511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.417372 (+1000861us) service_pool.cc:224] Handling call
0709 13:03:16.429905 (+ 12533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.584579 (+154674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.584800 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.559456 15235 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:16.587013 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:16.589443 17299 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.594056 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 147606us
I20260709 13:03:16.405581 17200 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.608144 17178 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:16.451700 17220 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:03:16.608711 17178 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:16.609252 17178 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:16.593994 17353 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260709 13:03:16.613418 15232 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:16.587690 17404 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.636108 15230 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:16.594411 17053 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:16.631963 15079 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:16.633462 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1747) took 4264 ms. Trace:
W20260709 13:03:16.649420 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 166532us
I20260709 13:03:16.625878 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.605530 17390 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:16.607257 17297 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.589998 17417 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:16.596695 17277 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.609416 17220 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:16.678222 15158 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:16.684327 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:16.676728 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50957.4us
W20260709 13:03:16.685860 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 224991us
I20260709 13:03:16.652299 15161 rpcz_store.cc:276] 0709 13:03:12.368987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.805469 (+436482us) service_pool.cc:224] Handling call
0709 13:03:12.805811 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.806142 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.806165 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.826996 (+ 20831us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.897128 (+1070132us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b1800261940. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:14.040440 (+143312us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:14.040483 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.040556 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.281648 (+2241092us) spinlock_profiling.cc:243] Waited 938 us on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:16.627425 (+345777us) raft_consensus.cc:1685] finished
0709 13:03:16.627467 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.633242 (+  5775us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21158,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":14296,"spinlock_wait_cycles":13870336,"thread_start_us":143091,"threads_started":1,"wal-append.queue_time_us":2223829}
I20260709 13:03:16.642951 17317 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:16.741336 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149565us
I20260709 13:03:16.642458 17404 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.736428 15234 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:16.753270 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.782166 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.615324 17299 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:16.686291 17200 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:16.691725 17391 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:16.683458 17417 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.752759 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.682374 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1751) took 1049 ms. Trace:
I20260709 13:03:16.738073 17220 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:16.789510 15082 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:16.795316 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.800549 17309 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:16.810312 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.811304 15160 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:16.788381 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 761) took 1290 ms. Trace:
I20260709 13:03:16.849871 15080 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:16.828099 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.815675 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.850579 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:16.821333 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.789094 17391 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:16.855970 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:16.856953 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54786.9us
W20260709 13:03:16.861259 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.798899 17314 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:16.863418 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.863883 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2216) took 1050 ms. Trace:
I20260709 13:03:16.863381 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:16.735186 17222 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.791731 17417 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:16.864926 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:16.808784 15843 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:16.866503 15080 rpcz_store.cc:276] 0709 13:03:15.801144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.801293 (+   149us) service_pool.cc:224] Handling call
0709 13:03:16.177626 (+376333us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:16.851929 (+674303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3173248}
W20260709 13:03:16.818876 17427 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:16.830396 17218 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.868637 17208 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:16.797554 15156 rpcz_store.cc:276] 0709 13:03:15.622079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.075767 (+453688us) service_pool.cc:224] Handling call
0709 13:03:16.076183 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.076471 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:03:16.097616 (+ 21145us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.581266 (+483650us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.584983 (+  3717us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:16.597107 (+ 12124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.611583 (+ 14476us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.652434 (+ 40851us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":363,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":3411,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8319,"thread_start_us":181388,"threads_started":1,"wal-append.queue_time_us":596350}
I20260709 13:03:16.869359 17390 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.870024 17400 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.870123 15086 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:16.871865 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1757) took 1078 ms. Trace:
I20260709 13:03:16.871994 15084 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.845101 17433 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.873426 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.829604 17106 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:16.876183 15082 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:16.920830 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:16.855940 17302 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.877300 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.921038 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:16.921862 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:16.922941 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.926290 16946 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:16.865727 17391 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:16.904145 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 251298us
I20260709 13:03:16.903412 15155 rpcz_store.cc:276] 0709 13:03:15.793224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.374894 (+581670us) service_pool.cc:224] Handling call
0709 13:03:16.411640 (+ 36746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.449065 (+ 37425us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:03:16.584619 (+135554us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.871439 (+286820us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.871526 (+    87us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:16.871578 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.871646 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.871781 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116702,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":3232,"thread_start_us":187801,"threads_started":1}
I20260709 13:03:16.877060 17285 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:16.905007 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.931881 15159 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:16.905157 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.908594 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.823203 17220 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.913069 17218 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:16.882870 17208 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.917090 17432 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.917346 17117 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.843s	user 0.010s	sys 0.001s
I20260709 13:03:16.842319 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:16.924098 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.791409 17297 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:16.928719 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:16.924265 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.928090 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2320) took 4566 ms. Trace:
W20260709 13:03:16.933120 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.903813 15233 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.004s
I20260709 13:03:16.850030 15841 rpcz_store.cc:276] 0709 13:03:15.491924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.515104 (+ 23180us) service_pool.cc:224] Handling call
0709 13:03:16.742204 (+1227100us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b540036ced4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:16.781133 (+ 38929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737409792}
I20260709 13:03:16.936652 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:16.921182 17423 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:03:16.983865 15162 rpcz_store.cc:276] 0709 13:03:12.361722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.741023 (+379301us) service_pool.cc:224] Handling call
0709 13:03:15.981794 (+3240771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.087104 (+105310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.925335 (+838231us) spinlock_profiling.cc:243] Waited 3.18 s on lock 0x7b54001c6bd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:16.928024 (+  2689us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7324558592}
I20260709 13:03:16.938845 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:16.984030 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44921.4us
W20260709 13:03:16.984930 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.937932 17208 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.925169 17338 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:16.988099 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.001431 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 125482us
W20260709 13:03:17.001902 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 151632us
I20260709 13:03:16.989351 16946 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:17.002549 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.985630 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:16.945191 15088 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:16.991468 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.998250 15084 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:16.989863 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43498.2us
I20260709 13:03:16.937901 17178 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.002710 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:16.989719 17353 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.006022 15083 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:16.907850 17390 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:16.984087 17106 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:16.940375 17431 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.022789 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:17.016142 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 83075.5us
I20260709 13:03:16.925540 17400 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:17.017860 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:16.924867 17314 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:17.023720 15084 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:17.024547 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:17.016927 17428 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:03:17.038610 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.039655 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.039839 17443 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:17.040304 17443 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:17.040472 15162 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:17.041141 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:16.932111 17433 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.042129 17329 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.042677 17329 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.043188 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:17.044061 17200 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.044232 17433 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.044739 17200 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:17.045888 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.048131 16998 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:17.049659 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:17.041792 16946 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.026811 17309 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:17.054014 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.054735 15157 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.041863 15163 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:17.055825 15158 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:16.936765 17302 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:17.063299 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.020558 17106 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:17.068168 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 132663us
I20260709 13:03:17.040704 17443 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:17.028182 17220 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.035439 17117 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:17.089025 17117 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.089465 17117 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.071620 17106 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:17.067684 17326 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.963s	user 0.008s	sys 0.003s
W20260709 13:03:17.081872 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 181210us
I20260709 13:03:17.045930 17178 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:17.046497 17418 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:17.082835 17314 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.083940 17433 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:17.071069 17293 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:17.070768 17053 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.103544 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:17.110948 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 74392us
I20260709 13:03:17.030951 15088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:17.125782 15088 raft_consensus.cc:606] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.112953 15229 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:17.029687 17431 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.010690 17432 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:17.117374 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44991.3us
I20260709 13:03:17.123340 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.046841 17299 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:17.032363 17353 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:17.128656 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59001.5us
I20260709 13:03:17.129581 17277 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:17.118355 17298 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:03:17.136725 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 106270us
W20260709 13:03:17.137044 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 134815us
I20260709 13:03:17.125582 17293 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:17.123921 17400 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:17.152649 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 216059us
W20260709 13:03:17.144265 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 113914us
I20260709 13:03:17.126969 17222 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:17.047107 17200 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.156107 17222 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.162942 17222 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.156358 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.165051 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.165897 15088 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:17.137759 17326 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:03:17.168108 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.168341 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.150624 17285 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:17.138038 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.169544 15086 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:17.170171 17053 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.170317 17431 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.135097 17106 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.173385 17277 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:17.174352 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.147985 15085 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:17.181491 15157 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:17.142719 17178 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:17.136163 17297 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.138600 17298 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:17.142620 17429 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:17.159688 17436 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:03:17.123538 17417 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.152668 17438 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:03:17.155504 17293 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:17.199158 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.199664 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.200309 17208 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.201140 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:17.206414 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:17.206955 17404 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.208457 17353 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.125479 16998 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:03:17.157980 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.370716 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:17.271816 15160 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:17.497423 15846 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:17.408308 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.497246 15080 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:17.252061 15158 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:03:17.158689 15841 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:03:17.462463 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.496785 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 448) took 1080 ms. Trace:
I20260709 13:03:17.505190 15157 rpcz_store.cc:276] 0709 13:03:16.416673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.053710 (+637037us) service_pool.cc:224] Handling call
0709 13:03:17.496716 (+443006us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:17.496143 15051 connection.cc:256] RPC call timeout handler was delayed by -1.6745565059999308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3267s for a response
W20260709 13:03:17.388108 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.219401 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:17.507409 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 288102us
I20260709 13:03:17.250630 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.403995 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1641) took 1326 ms. Trace:
I20260709 13:03:17.220708 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:17.509809 15229 rpcz_store.cc:276] 0709 13:03:16.077503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.084424 (+  6921us) service_pool.cc:224] Handling call
0709 13:03:17.360457 (+1276033us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:17.403869 (+ 43412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338780544}
I20260709 13:03:17.512728 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.292336 17445 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:17.219722 17446 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.326s
I20260709 13:03:17.515237 17233 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:17.515981 17233 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.516366 17233 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.211748 17391 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.475833 17285 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.518359 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:17.161167 17432 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.519071 17391 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:17.184263 17338 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:17.516243 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 348200us
I20260709 13:03:17.411022 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:17.507925 15081 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:17.522038 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 272076us
I20260709 13:03:17.515357 17445 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:17.321622 17442 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.280s	user 0.000s	sys 0.004s
W20260709 13:03:17.526170 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.534162 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 376728us
W20260709 13:03:17.528295 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.494192 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 359462us
I20260709 13:03:17.531709 15237 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 4. Ops: []
I20260709 13:03:17.320492 17451 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:17.539645 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.540434 15847 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:17.216423 17302 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:17.542142 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.542984 15157 raft_consensus.cc:2376] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2. Re-sending same reply.
I20260709 13:03:17.543143 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:17.503473 17329 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.545607 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.548170 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.524222 17178 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.550931 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.407727 17390 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:17.554502 15160 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:17.188337 17433 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.345s
I20260709 13:03:17.199689 17117 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.558308 15842 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:17.559267 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:17.217273 17317 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.327s
I20260709 13:03:17.571957 17400 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.552206 17277 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:17.573173 17277 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:17.573830 17053 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:17.574175 17277 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.575112 17338 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.579416 17448 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:17.580039 17448 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:17.580529 17448 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.504101 17222 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:17.582935 17222 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:17.533701 17445 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.587034 17117 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.541738 17418 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.559347 17178 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.589293 17418 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:17.593338 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 182461us
I20260709 13:03:17.594307 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:17.600911 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.606774 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 434) took 2093 ms. Trace:
I20260709 13:03:17.607432 15237 rpcz_store.cc:276] 0709 13:03:15.472353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.407545 (+935192us) service_pool.cc:224] Handling call
0709 13:03:16.433992 (+ 26447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.536366 (+1102374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.565304 (+ 28938us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400318b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:17.565554 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523911552}
I20260709 13:03:17.523648 15843 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
I20260709 13:03:17.542333 17353 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.612267 15846 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:17.618952 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 120232us
I20260709 13:03:17.621255 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.625288 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.626205 15235 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:17.629881 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.629858 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.631604 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.632547 15162 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:17.606681 17398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.035s	sys 0.040s
I20260709 13:03:17.220278 17444 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.384s
W20260709 13:03:17.641916 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47701.1us
I20260709 13:03:17.642994 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:17.277101 17447 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.326s
I20260709 13:03:17.637245 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.355332 17443 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.638437 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:17.637697 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 94616.9us
I20260709 13:03:17.635020 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.606297 17390 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:17.636164 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.653986 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.654261 15228 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
W20260709 13:03:17.654399 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.654584 15158 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:17.576659 17309 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:03:17.606554 17400 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:17.653107 15051 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 3.127s (SENT)
I20260709 13:03:17.653399 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.665277 15083 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:17.665138 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.655495 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.665930 15236 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:17.666390 15084 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:17.666795 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 129628us
I20260709 13:03:17.667820 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:17.618597 17451 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.655251 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:17.655292 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:17.644678 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:17.676218 15232 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:17.644553 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.677663 15229 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:17.652060 15155 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:17.639484 17398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.035s	sys 0.040s
I20260709 13:03:17.658340 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:17.679186 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.671782 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.673790 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.644960 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.680269 15080 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:17.680562 15234 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:17.658531 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.681121 15230 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:17.680365 15846 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:17.685783 15228 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:17.640637 17053 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.635725 17222 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:17.691267 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.635915 17200 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:17.678393 17390 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:17.694340 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.658202 17353 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:17.697129 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.718644 15155 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:03:17.719863 15085 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:17.697310 17400 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:17.697914 17463 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:17.723110 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.698017 17437 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:17.547426 17298 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.698516 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.699402 15230 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:17.699888 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.726689 15846 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:17.686496 17418 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:17.700454 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59139us
I20260709 13:03:17.709188 15839 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:17.695348 17446 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:17.723903 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:17.701000 17218 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:17.688031 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:17.730505 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.687376 17398 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:17.733053 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:17.735739 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.744812 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 73173.5us
W20260709 13:03:17.750131 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56796.5us
I20260709 13:03:17.722913 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.752437 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.752961 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.753854 15155 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:17.723520 17463 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.753979 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 98821.1us
I20260709 13:03:17.754727 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.756770 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:17.757532 15080 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:17.757663 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.751905 15081 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:17.755573 16946 raft_consensus.cc:2764] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:03:17.686332 17220 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 13:03:17.750198 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68138.7us
W20260709 13:03:17.763432 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.760447 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.764243 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:17.767059 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.736616 17446 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.744130 17418 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:17.755775 17438 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:17.796854 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.797631 15846 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:17.726323 17222 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:17.783322 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:17.784606 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 141687us
I20260709 13:03:17.773847 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.782048 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.803200 15155 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:17.790117 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.804116 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:17.791704 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:17.806257 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53362.4us
I20260709 13:03:17.806788 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:17.761807 17463 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:17.797880 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.787463 17117 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.782049 15232 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:17.831471 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.788707 17446 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.835225 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.688467 17451 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:17.736279 17200 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.929332 15080 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:17.930325 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:17.930341 15160 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:17.931756 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:17.789641 17470 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:17.936448 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:17.936657 15235 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:17.795745 17404 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.809329 17437 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:17.808321 17208 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.943617 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.950062 15084 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.950649 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:17.950744 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.951458 15081 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.951547 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:17.952422 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:17.789566 17452 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:03:17.956574 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.957468 15088 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.957929 15080 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:17.958828 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.959347 15086 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:17.936657 17418 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:17.957616 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:17.963369 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:17.954216 17457 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.418s	user 0.000s	sys 0.002s
I20260709 13:03:17.944303 17437 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:17.965713 17447 raft_consensus.cc:2764] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:03:17.960150 17472 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:17.965761 17452 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:17.967104 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.954491 17218 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:17.955024 17285 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:17.934314 17117 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:17.981170 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49942.9us
W20260709 13:03:17.981189 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.982303 15084 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:17.998947 15085 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:17.983819 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:17.954273 17390 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:17.987535 17437 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:17.989187 17467 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:03:17.991597 15233 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:17.993151 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:18.036089 15086 raft_consensus.cc:2376] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5. Re-sending same reply.
I20260709 13:03:17.993973 17417 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:17.964568 15843 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:17.999801 15087 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:17.966432 17404 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:17.985491 17465 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
I20260709 13:03:17.975158 15841 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:18.033988 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.034250 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.040802 15085 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:17.944653 17353 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:18.042387 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 260458us
I20260709 13:03:18.045397 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:17.954746 17398 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:18.049439 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 242699us
I20260709 13:03:18.029414 17447 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:18.002019 17472 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:17.975548 17418 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.026511 17443 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.025929 17470 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:18.037854 17178 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.109680 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:18.095580 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61683.5us
I20260709 13:03:18.037856 17220 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:18.079505 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:18.070117 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 106825us
I20260709 13:03:18.070040 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.138429 15085 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:18.133615 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 98849.6us
W20260709 13:03:18.053436 17293 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
W20260709 13:03:18.145582 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100259us
I20260709 13:03:18.146610 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:18.078135 17471 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.211s	user 0.000s	sys 0.002s
I20260709 13:03:18.082145 17437 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.053589 17447 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:18.096589 17404 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.180811 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.078634 17430 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:18.073171 17417 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:18.128978 17222 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:18.079018 17473 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.141331 17285 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:18.190850 15086 raft_consensus.cc:2376] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5. Re-sending same reply.
W20260709 13:03:18.181268 17442 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:18.114331 17477 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:18.223906 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.241142 15079 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:18.242537 15088 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:18.184237 17433 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.243678 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 109075us
I20260709 13:03:18.240645 15081 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:18.273741 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:18.148675 17474 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:18.177384 17293 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.234814 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:18.293576 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58828us
W20260709 13:03:18.255380 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.286888 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.287161 15088 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:18.287652 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:18.202441 17178 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.189895 17443 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:18.300707 15088 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:18.235528 17451 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:18.307492 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:18.273226 17208 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.223275 17404 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.276540 17473 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.145282 17472 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.203557 17200 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.241480 17313 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 13:03:18.428658 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 133985us
I20260709 13:03:18.500545 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:18.347563 15083 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:18.310303 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:18.413704 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:18.476361 15155 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:03:18.428390 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 154738us
W20260709 13:03:18.427577 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.533321 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:18.531944 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 221708us
I20260709 13:03:18.531052 15158 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:18.535720 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:18.535919 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:18.449162 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 260708us
I20260709 13:03:18.290027 17431 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:18.528860 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 297859us
I20260709 13:03:18.308363 17218 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:18.573264 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.477113 15087 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:18.278749 17477 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:18.416530 17053 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.703s	user 0.005s	sys 0.002s
I20260709 13:03:18.305047 17417 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:18.504925 17445 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.920s	user 0.011s	sys 0.002s
W20260709 13:03:18.344193 17233 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.827s	user 0.006s	sys 0.003s
W20260709 13:03:18.424187 17448 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.001s
W20260709 13:03:18.579773 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 273183us
W20260709 13:03:18.544431 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.419167 17479 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
W20260709 13:03:18.575107 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44021.8us
I20260709 13:03:18.294049 17433 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:18.418196 17483 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:03:18.417009 17469 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:18.345155 17451 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:18.568369 17208 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.589781 17313 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:18.417273 17480 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.357s
I20260709 13:03:18.341257 17478 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.385s
I20260709 13:03:18.339316 17293 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.341830 17418 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:18.480552 17443 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:18.476091 17220 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:18.575467 17431 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:18.595942 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.525754 17486 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:03:18.597045 17448 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.183708 17470 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.341022 17476 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.315s
I20260709 13:03:18.595857 17233 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.598083 17460 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:18.629479 17293 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:18.629742 17460 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:18.630373 17460 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.630903 17233 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.631470 17208 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.632200 17208 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.616290 17313 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:18.633579 17443 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.633723 17313 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.635491 17313 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.599052 17317 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:18.600744 17106 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.350261 17447 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:18.604228 17463 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
I20260709 13:03:18.586938 17053 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.622311 17469 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.348189 17314 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:18.626781 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:18.298734 17474 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.666220 17497 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.671232 17497 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.671712 17433 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.672019 17474 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.422540 17285 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.673435 17497 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.590690 17218 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.674432 17220 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.653447 17470 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.619030 17488 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:18.676964 17314 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:18.677541 17488 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:18.677973 17314 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.678577 17490 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:18.679106 17314 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.681210 17447 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.677992 17494 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:03:18.664445 17463 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.685108 17463 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:18.625247 17446 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.684s	user 0.003s	sys 0.008s
I20260709 13:03:18.620536 17451 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:18.685490 17404 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.686396 17451 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.687247 17448 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.662775 15842 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:18.657341 17317 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:18.678524 17488 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.685799 17446 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.585170 17417 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.593387 17445 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:18.723659 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 53829.2us
W20260709 13:03:18.729166 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100540us
W20260709 13:03:18.737500 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 138194us
W20260709 13:03:18.738425 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.740272 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:18.723769 17326 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:18.723111 17317 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:18.723385 17404 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.765640 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 232376us
W20260709 13:03:18.768775 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.773088 17473 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:18.773672 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.773975 17505 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:18.774478 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:18.693074 17477 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.733034 17496 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:18.731869 17293 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:03:18.778173 17496 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.736954 17417 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.780085 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:18.782207 15158 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:18.783702 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:18.768903 17437 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.003s
I20260709 13:03:18.767473 17326 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:18.723500 17469 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.794452 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.796355 15158 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:18.770833 17486 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.770757 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.797106 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:18.797147 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 223971us
I20260709 13:03:18.784382 17473 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:18.801649 15844 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:18.801882 17473 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:18.802533 17473 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:18.805243 17429 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.298s	user 0.006s	sys 0.007s
I20260709 13:03:18.805331 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 727) took 1029 ms. Trace:
I20260709 13:03:18.805791 17429 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.805840 15842 rpcz_store.cc:276] 0709 13:03:17.775622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.775771 (+   149us) service_pool.cc:224] Handling call
0709 13:03:17.790002 (+ 14231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.689051 (+899049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.805085 (+116034us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:18.805269 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006776192}
I20260709 13:03:18.786208 17480 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:18.781453 17496 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.798825 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:18.816166 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.828161 15158 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.794832 15843 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:18.861038 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.862977 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.782109 17404 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.865792 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.865242 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:18.866811 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44008.3us
I20260709 13:03:18.869540 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:18.815301 17277 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:18.871622 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.876730 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.771756 17431 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.885361 17431 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:18.866437 17222 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:18.879463 17485 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:18.866822 17470 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:18.901575 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:18.881731 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:18.882104 17492 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:18.882845 17491 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.882472 17326 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.883745 17504 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:18.907183 15848 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.891904 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.873168 17472 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.821941 17480 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:18.905395 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:18.826324 17398 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:03:18.831957 17313 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:03:18.871830 17390 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:18.892956 15082 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
W20260709 13:03:18.913679 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 146016us
I20260709 13:03:18.916200 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:18.954370 15847 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:18.916496 17472 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.905750 15158 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:18.966410 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:18.910763 17208 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:03:18.828006 17486 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:18.921875 17500 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.981456 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:18.922608 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:18.922848 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.983708 15128 leader_election.cc:336] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.673s (SENT)
I20260709 13:03:18.984380 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:18.983217 15845 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
W20260709 13:03:18.914698 17452 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.917s	user 0.013s	sys 0.000s
I20260709 13:03:18.997162 17452 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:18.998376 15155 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:18.924240 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 724) took 1767 ms. Trace:
I20260709 13:03:18.924829 17469 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.999253 15839 rpcz_store.cc:276] 0709 13:03:17.156650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.158737 (+  2087us) service_pool.cc:224] Handling call
0709 13:03:17.159053 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.744005 (+584952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.923914 (+1179909us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:18.924179 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264603520}
W20260709 13:03:18.929006 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.907568 17504 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:18.919184 17480 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.941308 17446 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:18.942605 17473 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:18.944962 15157 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:18.971609 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:19.019217 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47729.8us
I20260709 13:03:18.921120 17326 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:18.974535 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.022141 15847 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:18.977550 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2228) took 1199 ms. Trace:
I20260709 13:03:19.022490 15083 rpcz_store.cc:276] 0709 13:03:17.757936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.758090 (+   154us) service_pool.cc:224] Handling call
0709 13:03:17.758368 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.530847 (+772479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.956519 (+425672us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54001905d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:18.956784 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354409344}
I20260709 13:03:18.915441 17418 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:18.986562 17398 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.989281 17472 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:18.995731 17317 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:19.001825 15126 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.731s (SENT)
W20260709 13:03:19.014780 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126603us
W20260709 13:03:19.014523 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 276926us
I20260709 13:03:19.027885 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.013825 17504 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.015316 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.029996 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:19.030807 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 122130us
I20260709 13:03:19.028893 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:18.909374 17491 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.024772 17220 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:19.079396 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.062258 15848 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:19.078733 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.022367 17460 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.062265 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.087666 15230 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:19.058934 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.088235 15234 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:19.057845 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:19.040895 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.083321 15237 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:19.093421 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.085705 17473 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.058934 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:19.083963 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.095427 15842 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.095927 15848 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:19.097805 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68807.3us
I20260709 13:03:19.098728 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.099700 17293 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:19.100630 17293 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:19.101339 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.102530 17293 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:19.042634 15840 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.036087 17317 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.105129 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:19.042055 17519 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.034152 17404 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.039604 17472 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:19.105542 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64160.3us
I20260709 13:03:19.106002 15158 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:19.108402 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.111613 15845 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:19.089797 17313 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.110402 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:19.113071 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67725.2us
I20260709 13:03:19.113217 15230 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:19.112064 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.114738 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.115955 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.116734 15229 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.117200 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:19.096167 17452 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:19.105362 17430 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:19.120431 15848 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:03:19.120944 17524 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:19.121418 17522 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:19.122995 17508 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260709 13:03:19.123076 17491 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.125526 17326 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:19.104996 17432 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.165956 15847 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:19.129129 17404 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:03:19.120286 17523 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.078s	user 0.001s	sys 0.001s
I20260709 13:03:19.057578 17417 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.104460 17220 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.152264 17447 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.153575 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.154512 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.154876 17450 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:19.114017 17473 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:19.129864 17313 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.136404 17444 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:19.156411 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1765) took 1564 ms (client timeout 1986 ms). Trace:
I20260709 13:03:19.156930 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.157490 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:19.263522 15157 rpcz_store.cc:269] 0709 13:03:17.592441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.592819 (+   378us) service_pool.cc:224] Handling call
0709 13:03:19.156195 (+1563376us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540017eb54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.156323 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3041616768}
I20260709 13:03:19.264902 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:19.265565 15163 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:19.143134 17522 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:19.266459 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:19.157704 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.158443 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:19.158917 17499 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.356s	user 0.001s	sys 0.001s
I20260709 13:03:19.160138 15081 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:19.160728 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.166708 16946 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:19.273780 16946 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.273852 15842 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:19.234187 15084 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:19.177289 17106 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.176039 17233 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:19.236452 15083 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:19.166877 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.168018 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:19.167485 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64682.4us
I20260709 13:03:19.130399 17452 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:19.135769 17317 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.213317 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:19.168876 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.255102 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:19.292878 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.263753 15080 raft_consensus.cc:2376] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:19.113312 17519 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.294632 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.295291 15087 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:19.295517 17502 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:19.208249 17451 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.213693 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.176236 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:19.236045 17430 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:19.245980 17472 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.173283 15845 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:19.281540 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2348) took 1568 ms (client timeout 2043 ms). Trace:
W20260709 13:03:19.306615 15162 rpcz_store.cc:269] 0709 13:03:17.712929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.769291 (+ 56362us) service_pool.cc:224] Handling call
0709 13:03:19.281333 (+1512042us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001c6bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.281465 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363437312}
I20260709 13:03:19.248277 17447 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.284926 15230 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.286759 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.238118 17450 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.293026 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.287353 17106 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.321861 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.322666 17314 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:19.323400 17314 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:19.268304 15088 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.271652 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:19.174846 17496 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.325482 15163 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:03:19.296213 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 128276us
I20260709 13:03:19.269908 17444 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:19.298962 15157 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:03:19.287084 16946 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.299302 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.302281 15161 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:19.272531 17522 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.147217 17293 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 13:03:19.291939 17452 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.310985 15232 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.306226 17451 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.312826 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.313369 17472 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.315055 17430 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.316164 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.295948 17519 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.305966 17502 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.294670 17277 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:19.321543 17450 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.327332 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:19.296366 15086 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:19.366225 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:19.367501 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.323807 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.323766 15154 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:19.272991 17473 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.332129 15229 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:19.376631 15232 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:19.333727 15083 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:19.378000 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.323788 17314 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:19.334968 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.337062 16946 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.303139 17530 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:19.340620 17451 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.337138 17398 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:03:19.342737 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.344713 17472 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.283825 17233 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.374936 15088 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:19.380038 17519 raft_consensus.cc:2764] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:03:19.330319 17460 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.379285 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:19.472894 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93690.4us
I20260709 13:03:19.335116 17444 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.331465 17106 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.299460 17317 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.383821 15230 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:19.391717 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.304219 17329 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:03:19.349337 17514 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:19.396910 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.369231 15843 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:19.438972 17506 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:19.484741 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.486356 15230 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:19.491734 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 156843us
I20260709 13:03:19.335259 17220 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.376103 17293 raft_consensus.cc:2764] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:03:19.397680 17530 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:19.451197 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 280564us
I20260709 13:03:19.452159 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.499847 15232 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.453296 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1646) took 1827 ms. Trace:
W20260709 13:03:19.335547 17353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.040s	sys 0.053s
I20260709 13:03:19.376300 17505 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.394155 17314 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:19.411307 17431 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 13:03:19.459052 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.510501 15088 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:19.511528 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.512142 15088 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:19.465621 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.513299 15841 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:19.466709 17233 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:19.505045 17353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.041s	sys 0.053s
I20260709 13:03:19.514660 17233 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.471567 17496 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.438637 17450 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.492566 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.523784 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.495487 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.500893 15235 rpcz_store.cc:276] 0709 13:03:17.625922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.658109 (+ 32187us) service_pool.cc:224] Handling call
0709 13:03:19.453132 (+1795023us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400294854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.453248 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591462272}
I20260709 13:03:19.501405 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.524781 15230 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.481706 17477 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:19.439059 17504 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.445971 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:19.506085 15813 connection.cc:256] RPC call timeout handler was delayed by -1.1378753340000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:03:19.531038 15083 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:19.489377 17506 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:19.502429 17530 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.519688 15232 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:19.529989 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.537307 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:19.525679 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 357053us
I20260709 13:03:19.538916 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:19.527316 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:19.531069 15813 connection.cc:256] RPC call timeout handler was delayed by -1.0532753339999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 13:03:19.541244 15813 connection.cc:256] RPC call timeout handler was delayed by -1.0368829560001132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:03:19.506011 17480 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.534539 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.500780 17519 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:19.541872 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.537333 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.506512 17417 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:19.521456 17293 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:19.549729 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.549995 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.533442 17431 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.576063 15230 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:19.534997 17506 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.554313 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.569912 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:19.562186 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.537985 17329 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.569451 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.549484 17505 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.570549 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.550616 16946 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:19.569687 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.577646 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.585884 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.590102 15232 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:19.539860 17477 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.547173 17277 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:19.588403 15083 raft_consensus.cc:2368] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:19.569509 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.591415 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.538774 17398 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:03:19.595517 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.559087 17293 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:19.579365 17460 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.598335 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.600504 15232 raft_consensus.cc:2393] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:19.600288 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124014us
I20260709 13:03:19.582123 17507 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.607529 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.609200 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.620250 15088 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:19.621567 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.592736 17447 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.581214 17313 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.627036 15236 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:19.627291 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.641701 15230 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:19.603997 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 280229us
I20260709 13:03:19.628711 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.656507 15083 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:19.582551 17338 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:03:19.603049 17293 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.661761 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:19.630398 15840 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:03:19.626506 17485 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.684062 15236 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:19.666746 17277 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.640281 17507 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:19.662305 17353 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:19.667029 15083 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 4. Ops: []
I20260709 13:03:19.594655 17329 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.673066 17543 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:19.627820 17473 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.678591 17504 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:19.680104 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.703917 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.705008 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.639671 17505 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.706550 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 136088us
I20260709 13:03:19.707629 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.682798 17480 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.695158 15236 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:03:19.718858 15051 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.899s (SENT)
I20260709 13:03:19.597754 17514 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:19.697855 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 128011us
I20260709 13:03:19.697855 16946 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.721768 16946 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:19.730193 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133183us
I20260709 13:03:19.660826 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:19.732353 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.733322 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:19.702306 15230 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:19.685065 15840 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:19.693938 17502 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.737329 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.716499 17473 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:19.686350 17477 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:19.739822 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 82893.8us
I20260709 13:03:19.723276 17443 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:19.723058 17398 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.753314 15230 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:03:19.758873 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.722832 17505 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.743345 17298 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.748656 17537 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:19.772008 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1767) took 2149 ms (client timeout 1899 ms). Trace:
I20260709 13:03:19.754810 17480 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.760659 17338 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:19.772862 17537 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:19.764806 17443 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:19.776403 15233 rpcz_store.cc:269] 0709 13:03:17.622968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.641738 (+ 18770us) service_pool.cc:224] Handling call
0709 13:03:19.771737 (+2129999us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.771923 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467146624}
I20260709 13:03:19.778445 17507 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.795696 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:19.797304 15082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1617) took 2762 ms (client timeout 2741 ms). Trace:
W20260709 13:03:19.797725 15082 rpcz_store.cc:269] 0709 13:03:17.037894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.038287 (+   393us) service_pool.cc:224] Handling call
0709 13:03:19.797148 (+2758861us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540025ba54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.797241 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4064019328}
I20260709 13:03:19.798900 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.800015 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:19.785322 17547 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:03:19.805420 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.805976 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.806746 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.807361 15845 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:19.808002 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.789117 17537 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.808633 15848 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.814419 17329 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:19.815532 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.816735 15848 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:19.817204 17443 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.824657 17445 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.781798 17480 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.826756 17480 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.829995 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:19.833451 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:18.592605 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:19.835865 17498 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:19.837301 17498 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:19.838328 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:19.790805 17338 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:19.840615 17338 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.842900 17496 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:19.843327 17496 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:19.844799 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.827595 15234 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:03:19.846658 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:19.847293 15085 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:19.800755 15082 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:19.851519 17504 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:19.808561 17539 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:03:19.845779 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:19.868286 17314 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.873257 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.876104 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.824645 17505 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.892951 15085 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:19.864717 17452 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.861225 17445 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:19.903208 17522 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.003s
I20260709 13:03:19.903641 17522 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.904193 17452 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.904875 17293 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.905789 17522 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.907285 15234 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:19.868829 17178 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:03:19.909029 15234 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:19.911453 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.898674 17417 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:19.913008 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:19.899123 17496 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.923583 17496 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.927227 17314 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.862011 17504 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:19.906952 17537 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.893692 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:19.900552 15232 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:19.930401 15079 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:19.938036 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:19.779299 17514 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:19.928418 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:19.960114 17553 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:03:19.961102 15085 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:19.962389 17537 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:19.940320 17450 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:19.906989 17293 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.892335 17530 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.950309 17314 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.976824 15162 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:19.945596 17504 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:19.941393 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1768) took 2344 ms. Trace:
I20260709 13:03:19.981494 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:19.993377 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:19.982779 17507 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:19.984004 17233 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:19.985531 17431 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:19.985663 17486 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:19.987649 17530 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:19.990564 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 737) took 1124 ms. Trace:
I20260709 13:03:20.006544 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:20.006752 15842 rpcz_store.cc:276] 0709 13:03:18.866268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.881450 (+ 15182us) service_pool.cc:224] Handling call
0709 13:03:19.990426 (+1108976us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540041cdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:19.990515 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442480384}
I20260709 13:03:20.007215 15155 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:19.969808 17444 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:19.974699 17546 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:19.949187 17445 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:03:19.957762 15841 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:19.971872 17178 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.993312 17314 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.001744 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:20.013244 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:20.014709 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1768) took 2391 ms (client timeout 2033 ms). Trace:
W20260709 13:03:20.018204 15231 rpcz_store.cc:269] 0709 13:03:17.623287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.638139 (+ 14852us) service_pool.cc:224] Handling call
0709 13:03:20.009948 (+2371809us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003d90d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:20.014614 (+  4666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330078464}
I20260709 13:03:20.019721 15154 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:20.021389 15088 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:19.999984 17285 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:20.028339 15228 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:20.006232 17505 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.012087 17537 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.010617 17546 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.008266 17293 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:03:20.029243 17432 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:20.047704 15231 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:20.028184 15840 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:20.038827 17546 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:20.021826 17178 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.067373 17314 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.068037 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:20.068531 17486 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.068938 15079 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:20.038666 17537 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:20.069844 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:20.054250 17391 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.061142 17432 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:20.034363 17504 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.068992 17486 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:20.063684 17506 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.080631 17391 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.097996 17391 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:20.101480 17484 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:20.101982 17484 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:20.102936 17484 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.104274 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1775) took 1689 ms. Trace:
I20260709 13:03:19.999233 15156 rpcz_store.cc:276] 0709 13:03:17.597410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.605108 (+  7698us) service_pool.cc:224] Handling call
0709 13:03:17.712385 (+107277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.762968 (+ 50583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.773885 (+ 10917us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.527280 (+1753395us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b2c00000448. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:19.549000 (+ 21720us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.759117 (+210117us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:19.763618 (+  4501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.769807 (+  6189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.872485 (+102678us) raft_consensus.cc:1685] finished
0709 13:03:19.872510 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.939410 (+ 66900us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5719936,"thread_start_us":93576,"threads_started":1,"wal-append.queue_time_us":270197}
I20260709 13:03:20.431298 17546 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:20.120750 14982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 354) took 13089 ms (client timeout 15000 ms). Trace:
W20260709 13:03:20.238350 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.112692 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:20.141161 15079 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:20.114051 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 779) took 2279 ms (client timeout 1556 ms). Trace:
W20260709 13:03:20.442631 15846 rpcz_store.cc:269] 0709 13:03:17.835142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.936300 (+101158us) service_pool.cc:224] Handling call
0709 13:03:20.113850 (+2177550us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54006d1d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:20.113986 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909614976}
I20260709 13:03:20.115737 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:20.111409 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:20.176611 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.451434 15126 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 3.315s (SENT)
I20260709 13:03:20.444483 15082 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:20.452098 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:20.441901 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 358946us
I20260709 13:03:20.171124 15081 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:20.462121 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:20.464636 15228 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:20.461621 15083 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:20.470120 15081 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:20.466287 15086 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:20.104696 15159 rpcz_store.cc:276] 0709 13:03:18.414625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.414782 (+   157us) service_pool.cc:224] Handling call
0709 13:03:18.415123 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.673383 (+258260us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:18.673687 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.766795 (+ 93108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.766853 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:18.766896 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.122313 (+355417us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:20.104019 (+981706us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54001d8654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:20.104065 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.104194 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11762,"prepare.queue_time_us":7428,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":2617,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":9405,"spinlock_wait_cycles":610491520,"thread_start_us":92678,"threads_started":1,"wal-append.queue_time_us":122665,"wal-append.run_cpu_time_us":3329,"wal-append.run_wall_time_us":1373218}
W20260709 13:03:20.429005 17569 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 13:03:20.431378 17560 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.369s	user 0.003s	sys 0.000s
W20260709 13:03:20.215797 17564 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:03:20.475773 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.077668 17445 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.399s
I20260709 13:03:20.120507 17314 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.108675 17444 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.459744 17277 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.439785 17546 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:20.107137 17178 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:20.183174 17496 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.483662 17537 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:20.483832 17443 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.484948 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:20.477015 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:20.476214 17484 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:20.495534 17506 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:20.496417 17557 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.436s	user 0.000s	sys 0.000s
W20260709 13:03:20.496625 17558 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:03:20.494648 17277 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.490749 17314 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:20.438889 14982 rpcz_store.cc:269] 0709 13:03:07.035637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.217083 (+181446us) service_pool.cc:224] Handling call
0709 13:03:07.652537 (+435454us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:07.723407 (+ 70870us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5400002490. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8c0640 00007f0d45797b76 00007f0d458a1c3e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:20.120683 (+12397276us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:16.789340 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:16.805986 (+ 16646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:16.806046 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:03:16.806064 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:16.806480 (+   416us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:16.806618 (+   138us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:16.806646 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:03:16.806982 (+   336us) tablet.cc:701] Row locks acquired
0709 13:03:16.807035 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:03:16.807198 (+   163us) write_op.cc:270] Start()
0709 13:03:16.807342 (+   144us) write_op.cc:276] Timestamp: P: 1783602196807132 usec, L: 0
0709 13:03:16.807359 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:03:16.808170 (+   811us) log.cc:844] Serialized 539 byte log entry
0709 13:03:17.320566 (+512396us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:17.789285 (+468719us) op_driver.cc:464] REPLICATION: finished
0709 13:03:17.834289 (+ 45004us) write_op.cc:301] APPLY: starting
0709 13:03:17.834415 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:17.834573 (+   158us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:17.945741 (+111168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:18.043191 (+ 97450us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:18.108958 (+ 65767us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:03:18.109067 (+   109us) write_op.cc:312] APPLY: finished
0709 13:03:18.109448 (+   381us) log.cc:844] Serialized 23 byte log entry
0709 13:03:18.109661 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:18.109755 (+    94us) write_op.cc:454] Released schema lock
0709 13:03:18.109818 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:18.185295 (+ 75477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:18.188706 (+  3411us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15405,"reported_tablets":5,"spinlock_wait_cycles":118965376,"child_traces":[["op",{"apply.queue_time_us":16297,"apply.run_cpu_time_us":3374,"apply.run_wall_time_us":292622,"num_ops":1,"prepare.queue_time_us":688248,"prepare.run_cpu_time_us":59277,"prepare.run_wall_time_us":268475,"raft.queue_time_us":165825,"raft.run_cpu_time_us":24521,"raft.run_wall_time_us":178567,"replication_time_us":965073,"spinlock_wait_cycles":49097984,"thread_start_us":459094,"threads_started":4,"wal-append.queue_time_us":173309,"wal-append.run_cpu_time_us":28787,"wal-append.run_wall_time_us":2265164}]]}
I20260709 13:03:20.504267 17220 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:20.493919 17443 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:20.109575 17432 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.502527 17537 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:20.510515 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 803970us
I20260709 13:03:20.348675 17486 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.514484 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 470) took 1261 ms. Trace:
I20260709 13:03:20.515069 17117 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.515657 17277 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.521579 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:20.501673 17496 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:20.564677 15128 leader_election.cc:400] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4.
I20260709 13:03:20.566196 15084 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:20.566280 15230 rpcz_store.cc:276] 0709 13:03:19.253503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.769741 (+516238us) service_pool.cc:224] Handling call
0709 13:03:19.782871 (+ 13130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.786993 (+  4122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.787024 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.866551 (+ 79527us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.062892 (+196341us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:20.062950 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.063458 (+   508us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.514230 (+450772us) raft_consensus.cc:1685] finished
0709 13:03:20.514251 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.514418 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2292,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":2196,"thread_start_us":126438,"threads_started":1,"wal-append.queue_time_us":587398}
I20260709 13:03:20.570489 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 472) took 1284 ms. Trace:
I20260709 13:03:20.573362 17443 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:20.579128 17566 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:03:20.517246 17220 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.580585 15082 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:20.563802 15160 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.033s	sys 0.067s
I20260709 13:03:20.588980 15233 rpcz_store.cc:276] 0709 13:03:19.285869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.794961 (+509092us) service_pool.cc:224] Handling call
0709 13:03:19.795325 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.795712 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.795744 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.796244 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.931872 (+135628us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.979813 (+ 47941us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.989515 (+  9702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.989636 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.495775 (+506139us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:20.522676 (+ 26901us) raft_consensus.cc:1685] finished
0709 13:03:20.561223 (+ 38547us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.565996 (+  4773us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35422,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":1560,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":194,"spinlock_wait_cycles":3439360,"thread_start_us":98596,"threads_started":1,"wal-append.queue_time_us":219724}
I20260709 13:03:20.593798 15087 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:20.587853 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:20.609509 15160 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.034s	sys 0.068s
I20260709 13:03:20.619635 17220 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:20.642423 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.643786 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:20.644313 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.626192 17443 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.650161 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:20.659008 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:20.660903 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.661248 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.671525 15847 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:20.671643 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:20.661883 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.665688 15841 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:20.666386 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.695451 15844 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:20.697084 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 245703us
I20260709 13:03:20.702533 15082 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:20.707072 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 222219us
I20260709 13:03:20.710806 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1783) took 1417 ms. Trace:
I20260709 13:03:20.711284 15162 rpcz_store.cc:276] 0709 13:03:19.293061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.307729 (+ 14668us) service_pool.cc:224] Handling call
0709 13:03:19.314361 (+  6632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.980336 (+665975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.710557 (+730221us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540017d754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:20.710745 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522843520}
I20260709 13:03:20.711843 15236 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:20.715260 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:20.716670 17418 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.717455 17418 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.718236 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:20.719295 15087 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:20.719710 17418 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.716619 17444 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:20.723764 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:20.724588 15082 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.722940 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:20.730906 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:20.733927 15236 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:20.739068 15236 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:20.740502 15231 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:03:20.743445 15236 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.745050 15236 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:20.749457 17496 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:20.754175 15230 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:20.763346 15082 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:20.797425 15235 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:20.729691 17314 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.750331 17470 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:20.769565 17445 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:20.729417 17444 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.805264 17470 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:20.827685 17470 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:20.823962 17585 raft_consensus.cc:2764] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:03:20.829301 17444 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.831432 17317 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:20.833278 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 161712us
W20260709 13:03:20.804477 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73646.6us
I20260709 13:03:20.840044 17578 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:20.821055 17277 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.841197 17575 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:20.841681 17575 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.841552 17277 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.842093 17575 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:20.843896 17277 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:20.845093 17477 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:20.816661 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 228872us
I20260709 13:03:20.729950 17418 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:03:20.848578 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:20.825348 17220 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.859321 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:20.854947 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:20.847832 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:20.857627 15160 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:20.865777 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:20.867167 17391 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:20.863832 15236 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:20.868618 15236 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
I20260709 13:03:20.869035 15236 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:03:20.869613 15236 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:20.844506 17277 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:03:20.867980 17391 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.874336 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:20.880229 15160 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:20.820752 17506 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:20.886667 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:20.818397 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 781) took 1014 ms. Trace:
I20260709 13:03:20.887972 15840 rpcz_store.cc:276] 0709 13:03:19.804132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.805705 (+  1573us) service_pool.cc:224] Handling call
0709 13:03:20.818197 (+1012492us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54006cdc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:20.818325 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489447808}
I20260709 13:03:20.873019 17317 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:20.889907 17317 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:20.890336 17317 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:20.872581 17314 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:03:20.894506 17220 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:20.888396 15162 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:20.896420 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:20.897024 15161 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:20.908385 15847 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:20.878644 17233 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:20.920944 17391 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:20.921346 15233 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:20.921643 17491 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:20.922897 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:20.923650 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:20.924469 15846 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:20.896124 17506 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.929307 17506 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.932046 15084 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:20.923694 15235 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:20.945667 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.947307 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 300486us
I20260709 13:03:20.923669 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:20.878868 17585 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:03:20.967793 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:20.971058 17575 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:20.972306 17317 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:20.967877 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.942589 17418 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:20.953876 17417 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:21.002727 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:21.003150 17444 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:21.003564 15233 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:21.029588 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.038754 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 173054us
W20260709 13:03:20.995671 15203 connection.cc:256] RPC call timeout handler was delayed by -1.6200036489999547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 13:03:20.998497 17431 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:21.043871 15235 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:20.986223 17418 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:21.085009 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:21.101854 16946 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.102519 16946 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:21.104099 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 269468us
I20260709 13:03:21.105705 15847 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:21.105878 17277 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:21.106246 17277 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.106572 17277 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.038877 17582 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.039132 17417 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:21.269273 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.608s (SENT) [suppressed 43 similar messages]
W20260709 13:03:21.430917 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 390707us
I20260709 13:03:21.430784 17429 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.441442 17429 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:21.432479 17443 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
W20260709 13:03:21.130554 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 179804us
I20260709 13:03:21.340545 17418 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.444448 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:21.101346 17220 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.344s
I20260709 13:03:21.445176 15230 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:21.445966 15082 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:21.227984 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:21.107743 17477 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:21.448004 17477 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:21.448369 17477 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:21.448628 15082 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.203871 17470 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.450997 15237 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:21.449689 17582 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.443734 17429 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:21.109406 17570 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:03:21.431780 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.078231 17431 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.319178 16946 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:21.453769 15230 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
W20260709 13:03:21.139091 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 286286us
W20260709 13:03:21.457856 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.402225 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.186582 17444 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.462262 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:21.481158 15232 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.030s	sys 0.026s
W20260709 13:03:21.481873 15232 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.031s	sys 0.026s
I20260709 13:03:21.452878 17431 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.458070 15082 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:21.150027 17587 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.517700 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:21.454442 17470 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:21.518368 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:21.454698 17537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.038s
W20260709 13:03:21.520809 15236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.021s	sys 0.023s
W20260709 13:03:21.517179 17576 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.300s	user 0.000s	sys 0.017s
I20260709 13:03:21.520558 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:21.437961 17314 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:21.522184 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:21.526331 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:21.522840 15236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.021s	sys 0.023s
I20260709 13:03:21.527913 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:21.462532 17485 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:21.530540 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.534879 16946 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:21.522855 17418 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:21.522053 15841 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.000s
W20260709 13:03:21.524441 17537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.038s
I20260709 13:03:21.517431 17575 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:21.593708 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.528438 17298 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.600004 16946 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:21.600488 16946 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:21.601338 16946 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:21.596360 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 491184us
I20260709 13:03:21.525949 17314 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.607771 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.506681 17444 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.538702 17178 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:21.520012 17587 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.581144 15156 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:03:21.590353 17429 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:21.644668 17429 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.600809 17317 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:21.534358 17548 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:03:21.645145 17429 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:21.592715 15841 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:21.651943 15233 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:21.614571 17314 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.654428 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2245) took 2178 ms. Trace:
I20260709 13:03:21.625190 17598 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:21.655645 15234 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:21.659350 15088 rpcz_store.cc:276] 0709 13:03:19.476025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.642233 (+166208us) service_pool.cc:224] Handling call
0709 13:03:21.649746 (+2007513us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400306e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:21.654353 (+  4607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817079168}
I20260709 13:03:21.640409 15156 pending_rounds.cc:85] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 4
I20260709 13:03:21.631330 17178 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:21.670984 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 442989us
I20260709 13:03:21.609444 17432 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:21.615979 17298 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:21.687721 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 418463us
I20260709 13:03:21.564625 17485 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:21.690793 15161 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.001s
I20260709 13:03:21.667593 15156 pending_rounds.cc:107] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 13:03:21.693917 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.694242 15161 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:21.696187 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.697119 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:21.678126 17178 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:21.665500 17598 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:21.724416 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 98778.9us
W20260709 13:03:21.715570 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 253479us
I20260709 13:03:21.732060 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:21.738286 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.739228 15088 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:21.740538 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.682159 17606 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:21.710974 15156 raft_consensus.cc:2889] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:21.746798 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:21.747938 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.741011 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:21.752936 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.754135 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.754814 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.755544 15087 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:21.744591 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:21.639740 17417 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.750741 17582 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:21.764400 17506 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.834s	user 0.008s	sys 0.000s
I20260709 13:03:21.768252 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:21.774914 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:21.775804 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:21.730716 17317 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:21.778235 17298 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:21.779536 17485 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:21.780156 15156 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:21.747424 17537 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:21.801281 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.789212 17607 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.833746 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:21.834678 15155 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:21.840508 15845 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:21.805373 17506 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.819933 17298 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:21.824374 17606 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:21.812888 17628 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:21.819173 17622 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:21.877018 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.872315 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.813443 15841 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:21.889025 15813 leader_election.cc:336] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.922s (SENT)
I20260709 13:03:21.903312 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:21.892517 17277 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.000s
W20260709 13:03:21.892796 15051 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.033s (SENT)
I20260709 13:03:21.819867 17592 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:03:21.901420 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.867431 17618 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.908926 17298 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:21.912407 17498 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.912634 17277 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.913165 17619 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:21.908989 15051 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.148s (SENT)
I20260709 13:03:21.825690 17537 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:21.928514 15813 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.961s (SENT)
I20260709 13:03:21.869237 17621 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.841315 17485 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:03:21.869343 17444 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:21.937420 17593 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
W20260709 13:03:21.925761 17612 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:03:21.948108 17627 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:03:21.959648 17314 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.947676 17445 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:21.843674 17607 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:21.963997 17625 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:21.937077 17609 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:21.965629 17587 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:21.967803 15161 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:21.901861 17590 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:21.877681 17585 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:21.865496 17522 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 13:03:21.973920 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64957.8us
W20260709 13:03:22.029773 15051 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.227s (SENT)
I20260709 13:03:22.030423 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:21.975946 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:21.980613 17506 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:21.948047 17498 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.975399 17621 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:21.983309 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:21.983662 17623 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:21.983863 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:21.854746 17598 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.989511 15234 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:21.990232 17548 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:21.953444 17618 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.017518 17277 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.012332 17553 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:22.017076 17298 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:21.908349 17622 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:22.013052 17443 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:21.948029 17619 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.016894 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.027603 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 502) took 1052 ms. Trace:
I20260709 13:03:22.036616 17285 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:22.062410 17608 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:03:22.081123 15232 rpcz_store.cc:276] 0709 13:03:20.975161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.975764 (+   603us) service_pool.cc:224] Handling call
0709 13:03:20.976196 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.482381 (+506185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.482428 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.488940 (+  6512us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.656098 (+167158us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:21.656532 (+   434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.656773 (+   241us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.027247 (+370474us) raft_consensus.cc:1685] finished
0709 13:03:22.027270 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.027511 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28969,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":626801,"threads_started":2,"wal-append.queue_time_us":248035}
I20260709 13:03:22.065532 17485 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:22.058568 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74765.5us
I20260709 13:03:21.908174 17606 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.038159 17430 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.037375 17314 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.081386 15235 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:22.057082 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 282252us
I20260709 13:03:22.038484 17611 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:22.092163 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:22.093282 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 396219us
I20260709 13:03:22.051980 17590 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:22.121968 15160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.008s	sys 0.028s
W20260709 13:03:22.182902 15160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.008s	sys 0.028s
I20260709 13:03:22.124450 15232 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.128414 15088 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:22.131026 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.131503 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.132906 15162 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:03:22.120492 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 389211us
I20260709 13:03:22.210431 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 500) took 1236 ms. Trace:
I20260709 13:03:22.210992 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:22.210861 15229 rpcz_store.cc:276] 0709 13:03:20.974459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.996989 (+ 22530us) service_pool.cc:224] Handling call
0709 13:03:21.046271 (+ 49282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:21.046745 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.046786 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:21.433056 (+386270us) log.cc:844] Serialized 52 byte log entry
0709 13:03:21.669697 (+236641us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.670213 (+   516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.670764 (+   551us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.210152 (+539388us) raft_consensus.cc:1685] finished
0709 13:03:22.210171 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.210351 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":893,"raft.queue_time_us":29145,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":17561,"thread_start_us":148134,"threads_started":1,"wal-append.queue_time_us":318709}
W20260709 13:03:22.152954 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 653) took 2185 ms (client timeout 1961 ms). Trace:
W20260709 13:03:22.215437 15079 rpcz_store.cc:269] 0709 13:03:19.967462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.069597 (+102135us) service_pool.cc:224] Handling call
0709 13:03:22.152620 (+2083023us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b54001b17d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:22.152815 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143776512}
W20260709 13:03:22.216094 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 471714us
I20260709 13:03:22.190459 15162 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:22.146522 15159 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:22.085150 17619 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:22.159364 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:22.167886 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.085443 17277 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:22.179736 16946 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.239010 15232 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:22.180388 15158 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:22.241326 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 486) took 1035 ms. Trace:
I20260709 13:03:22.241740 15158 rpcz_store.cc:276] 0709 13:03:21.205748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.205928 (+   180us) service_pool.cc:224] Handling call
0709 13:03:22.148586 (+942658us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:22.179975 (+ 31389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.240979 (+ 61004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.241254 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63951616}
I20260709 13:03:22.180671 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.246431 15231 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:22.149813 15163 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:22.216953 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:22.119951 17609 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:22.219182 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 487854us
I20260709 13:03:22.220860 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 492) took 1487 ms. Trace:
I20260709 13:03:22.165607 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 497) took 1258 ms. Trace:
I20260709 13:03:22.126577 17590 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:22.140156 17624 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
W20260709 13:03:22.289065 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.119788 17610 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:22.201115 17498 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.090483 17178 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.065028 17623 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.244771 15162 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.252354 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:22.251683 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 487) took 1037 ms. Trace:
I20260709 13:03:22.280303 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:22.281718 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70795.6us
I20260709 13:03:22.252691 15237 rpcz_store.cc:276] 0709 13:03:20.906789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.913002 (+  6213us) service_pool.cc:224] Handling call
0709 13:03:20.924180 (+ 11178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.451369 (+527189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.165200 (+713831us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b5400058cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:22.165486 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211148672}
W20260709 13:03:22.272789 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55941.5us
I20260709 13:03:22.252527 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.302423 15232 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:22.167718 17611 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.252110 15228 rpcz_store.cc:276] 0709 13:03:20.737047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.737195 (+   148us) service_pool.cc:224] Handling call
0709 13:03:20.737651 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.935349 (+197698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.935393 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.938073 (+  2680us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.428710 (+490637us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:21.428764 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.428879 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.220638 (+791759us) raft_consensus.cc:1685] finished
0709 13:03:22.220657 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.220782 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91645,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":160,"thread_start_us":677664,"threads_started":2,"wal-append.queue_time_us":719875}
I20260709 13:03:22.237452 17470 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:22.221563 17537 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:22.196765 17443 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:03:22.301267 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.318022 15231 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:22.237941 17277 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:22.321425 15084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.032s	sys 0.054s
W20260709 13:03:22.321817 15084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.032s	sys 0.054s
I20260709 13:03:22.264642 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.294407 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.323014 15233 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:22.323107 15229 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.248152 17607 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.207566 17547 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:03:22.251464 17631 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:22.262039 17496 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.192346 17430 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:22.304576 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.299443 15162 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:22.299897 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47630.3us
I20260709 13:03:22.299048 17636 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.287837 17293 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.300829 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:22.302071 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:22.381582 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 793) took 1276 ms. Trace:
W20260709 13:03:22.382154 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80185.2us
I20260709 13:03:22.312949 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.382229 15846 rpcz_store.cc:276] 0709 13:03:21.105050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.114216 (+  9166us) service_pool.cc:224] Handling call
0709 13:03:21.204568 (+ 90352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.388967 (+184399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.389013 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.429101 (+ 40088us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.697222 (+268121us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.697282 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.697380 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.139624 (+442244us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b24000d0cb8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c77d9c4 00007f0d3c77bacb 00007f0d3c936dc9 00007f0d3c936a55 00007f0d3c93609d 00007f0d3f56810e 00007f0d3f57f88c 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221
0709 13:03:22.141830 (+  2206us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4800260058. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:03:22.381267 (+239437us) raft_consensus.cc:1685] finished
0709 13:03:22.381296 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.381495 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":234238,"raft.queue_time_us":200100,"raft.run_cpu_time_us":3719,"raft.run_wall_time_us":713607,"spinlock_wait_cycles":71091840,"thread_start_us":179438,"threads_started":1,"wal-append.queue_time_us":340630}
I20260709 13:03:22.384405 15158 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:22.171360 17486 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.319816 16946 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.300004 17590 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:22.304255 17506 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:22.314030 17431 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.002s
W20260709 13:03:22.294487 17429 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.003s
I20260709 13:03:22.410473 15158 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:22.301826 17498 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:22.356325 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.356115 17496 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.332667 17548 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.326855 17277 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:22.358947 17606 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.335379 17631 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:22.360404 17622 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.361443 17430 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.361944 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:22.446326 17617 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:03:22.365435 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.366685 17443 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:03:22.369540 17630 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:03:22.447558 17609 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.447676 17443 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:22.448208 17609 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:22.370357 17635 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:03:22.449283 17443 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:22.449927 17614 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.757s	user 0.000s	sys 0.004s
I20260709 13:03:22.450037 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 495) took 1544 ms. Trace:
I20260709 13:03:22.308986 17603 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.310791 17338 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.383992 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:22.300554 15163 rpcz_store.cc:276] 0709 13:03:21.214021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.214222 (+   201us) service_pool.cc:224] Handling call
0709 13:03:22.149335 (+935113us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:22.149554 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.251397 (+101843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.251612 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25499904}
I20260709 13:03:22.385454 16946 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.386688 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:22.457826 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 73917.6us
I20260709 13:03:22.458375 15231 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:22.396334 15235 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:22.450523 15236 rpcz_store.cc:276] 0709 13:03:20.906080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.916254 (+ 10174us) service_pool.cc:224] Handling call
0709 13:03:20.917546 (+  1292us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:21.577603 (+660057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.590544 (+ 12941us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:21.600054 (+  9510us) log.cc:844] Serialized 52 byte log entry
0709 13:03:21.863653 (+263599us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.926882 (+ 63229us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.927119 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.927218 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.449806 (+522588us) raft_consensus.cc:1685] finished
0709 13:03:22.449827 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.449976 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62932,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4389,"raft.queue_time_us":134460,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":33632,"thread_start_us":829370,"threads_started":2,"wal-append.queue_time_us":446498}
I20260709 13:03:22.459816 15235 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:22.396930 15162 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:22.461102 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 483) took 1544 ms. Trace:
I20260709 13:03:22.457979 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:22.461351 15162 rpcz_store.cc:276] 0709 13:03:20.916798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.924799 (+  8001us) service_pool.cc:224] Handling call
0709 13:03:20.931264 (+  6465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.132742 (+1201478us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540015d220. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:03:22.460755 (+328013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.461042 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403556096}
I20260709 13:03:22.461742 15163 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:22.462329 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.447741 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 794) took 1333 ms. Trace:
I20260709 13:03:22.322254 17491 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.411592 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 110816us
I20260709 13:03:22.320555 17504 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.325469 17285 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.332234 17507 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.357566 15848 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:22.463126 15847 rpcz_store.cc:276] 0709 13:03:21.114704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.115483 (+   779us) service_pool.cc:224] Handling call
0709 13:03:21.115901 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.116193 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.116229 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.116754 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.660647 (+543893us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:21.660686 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.660755 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.447519 (+786764us) raft_consensus.cc:1685] finished
0709 13:03:22.447540 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.447682 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":219911,"threads_started":1,"wal-append.queue_time_us":863933}
I20260709 13:03:22.412796 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:22.395936 17599 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.363s	user 0.001s	sys 0.001s
I20260709 13:03:22.579037 17637 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.579734 17637 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.584026 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:22.584969 15079 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:22.585816 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:22.586407 15079 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:22.420567 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.411947 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:22.590121 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 178267us
I20260709 13:03:22.357560 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:22.593449 17578 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:22.360100 17623 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.398669 17636 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.596733 17636 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.400053 17506 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.597445 17506 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:22.430732 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.601198 15843 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:22.430932 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.408303 17293 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.317114 17611 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.433231 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.435732 17619 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:22.412364 17602 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:03:22.409169 17314 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:22.438541 17496 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.408959 17607 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.445079 17598 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.444905 17548 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.451184 17443 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:22.455608 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.398689 15233 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:22.817220 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1676) took 1356 ms. Trace:
I20260709 13:03:22.817580 15233 rpcz_store.cc:276] 0709 13:03:21.460560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.260012 (+799452us) service_pool.cc:224] Handling call
0709 13:03:22.817160 (+557148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.452586 15845 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEARNER]: Advancing to term 4
I20260709 13:03:22.470837 15229 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:22.818763 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1678) took 1357 ms. Trace:
I20260709 13:03:22.819078 15229 rpcz_store.cc:276] 0709 13:03:21.461003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.260698 (+799695us) service_pool.cc:224] Handling call
0709 13:03:22.818708 (+558010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.478353 17641 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:22.478044 17603 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.447254 17631 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.520483 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1670) took 1061 ms. Trace:
I20260709 13:03:22.557839 15159 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:22.451568 17178 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:22.828475 17633 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:03:22.827361 15230 rpcz_store.cc:276] 0709 13:03:21.458893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.459199 (+   306us) service_pool.cc:224] Handling call
0709 13:03:21.459682 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.715852 (+256170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.716894 (+  1042us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.734152 (+ 17258us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.937414 (+203262us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b18000ef380. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:22.378091 (+440677us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:22.378153 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.378257 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.520050 (+141793us) raft_consensus.cc:1685] finished
0709 13:03:22.520081 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.520410 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28236,"raft.queue_time_us":305912,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":29908,"spinlock_wait_cycles":22604032,"thread_start_us":668720,"threads_started":2,"wal-append.queue_time_us":745707}
I20260709 13:03:22.828903 17178 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.829351 17645 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:22.829588 17178 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.830621 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2252) took 1814 ms. Trace:
I20260709 13:03:22.831012 15083 rpcz_store.cc:276] 0709 13:03:21.016654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.017161 (+   507us) service_pool.cc:224] Handling call
0709 13:03:21.017567 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.078381 (+ 60814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.078410 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.078827 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.981526 (+902699us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18002f0b00. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:22.413454 (+431928us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:22.413497 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.413562 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.830339 (+416777us) raft_consensus.cc:1685] finished
0709 13:03:22.830358 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.830567 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":197723,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":112973,"spinlock_wait_cycles":28943104,"thread_start_us":431737,"threads_started":1,"wal-append.queue_time_us":1574817}
I20260709 13:03:22.832374 17178 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:22.415290 17429 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.848373 17429 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.509955 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.849148 15158 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:22.848968 17638 consensus_queue.cc:1175] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 13:03:22.850148 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:22.850677 15158 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:03:22.511780 17338 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.854389 17338 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.420603 17537 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:03:22.508288 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48931.2us
I20260709 13:03:22.857885 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.858639 15235 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:22.858861 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 657) took 2888 ms. Trace:
I20260709 13:03:22.859529 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.517275 17507 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.860674 15228 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:22.860965 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 797) took 1093 ms. Trace:
I20260709 13:03:22.512898 17491 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.509450 17285 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:22.525843 17594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:03:22.413734 17431 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.525220 16946 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:22.526654 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2251) took 1510 ms. Trace:
W20260709 13:03:22.532604 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67983.6us
I20260709 13:03:22.546748 15163 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:22.516104 17606 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:22.522764 17277 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.893741 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:22.661831 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57926.4us
I20260709 13:03:22.608002 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:22.859393 15086 rpcz_store.cc:276] 0709 13:03:19.970369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.471555 (+501186us) service_pool.cc:224] Handling call
0709 13:03:20.480223 (+  8668us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:20.487316 (+  7093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.487361 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:20.491292 (+  3931us) log.cc:844] Serialized 52 byte log entry
0709 13:03:22.123483 (+1632191us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b18003bdd60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:22.556778 (+433295us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:22.556866 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.556989 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.858472 (+301483us) raft_consensus.cc:1685] finished
0709 13:03:22.858499 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.858789 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6697,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4514,"spinlock_wait_cycles":9898240,"thread_start_us":433008,"threads_started":1,"wal-append.queue_time_us":2280129}
I20260709 13:03:22.574811 17498 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.572784 17601 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.689271 17605 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.903141 15162 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:22.692408 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2253) took 1675 ms. Trace:
I20260709 13:03:22.663710 17590 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:22.693208 15080 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:22.577145 17621 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.572372 17597 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:03:22.734134 15161 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:22.734716 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:22.739748 15201 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.020s	sys 0.021s
W20260709 13:03:22.916872 15201 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.020s	sys 0.021s
W20260709 13:03:22.917011 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 182399us
I20260709 13:03:22.726186 17592 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:22.777706 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.726351 17522 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:03:22.682624 17582 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:03:22.724445 17618 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.655634 15079 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:22.802342 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.808560 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.827054 17607 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:22.810364 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.940253 16946 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.940999 16946 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.943073 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.944360 15847 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:22.921078 17575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.002s	sys 0.016s
I20260709 13:03:22.822916 15231 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:22.793583 17477 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.345s	user 0.015s	sys 0.000s
I20260709 13:03:22.828070 17619 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.833158 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.510691 17504 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.828456 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.845718 17631 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:22.845861 17490 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.861326 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.890753 15844 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:22.965813 15236 raft_consensus.cc:2368] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:22.965915 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:22.889309 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.891381 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.967692 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.967865 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.968711 15230 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:22.891681 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.889525 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:22.861399 15842 rpcz_store.cc:276] 0709 13:03:21.767515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.767643 (+   128us) service_pool.cc:224] Handling call
0709 13:03:21.768011 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.768432 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.768469 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.770799 (+  2330us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.297140 (+526341us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:22.297183 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.297249 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.860737 (+563488us) raft_consensus.cc:1685] finished
0709 13:03:22.860756 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.860909 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":704,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":7952,"thread_start_us":482927,"threads_started":1,"wal-append.queue_time_us":940611}
I20260709 13:03:22.970319 15233 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:22.970427 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.971222 15842 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:22.661259 17293 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.890816 15084 rpcz_store.cc:276] 0709 13:03:21.016273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.018586 (+  2313us) service_pool.cc:224] Handling call
0709 13:03:21.077431 (+ 58845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.077698 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.077733 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.093901 (+ 16168us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.322265 (+1228364us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:22.322315 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.322399 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.526413 (+204014us) raft_consensus.cc:1685] finished
0709 13:03:22.526433 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.526579 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1105424,"threads_started":1,"wal-append.queue_time_us":1370774}
I20260709 13:03:22.900017 15082 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:22.611769 17637 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.904253 15081 rpcz_store.cc:276] 0709 13:03:21.016993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.017854 (+   861us) service_pool.cc:224] Handling call
0709 13:03:21.076272 (+ 58418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.076627 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.076652 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.212820 (+136168us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.964524 (+751704us) spinlock_profiling.cc:243] Waited 980 us on lock 0x7b180047b8a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:22.294745 (+330221us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:22.295210 (+   465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.295281 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.692046 (+396765us) raft_consensus.cc:1685] finished
0709 13:03:22.692072 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.692336 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":262077,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":91008,"spinlock_wait_cycles":2254720,"thread_start_us":330014,"threads_started":1,"wal-append.queue_time_us":1350000}
W20260709 13:03:22.898772 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3608438090000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:03:22.908418 15158 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:22.909044 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:22.983790 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:22.984180 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 392546us
I20260709 13:03:22.877483 14984 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 3, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 3 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:22.917229 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 795454us
W20260709 13:03:22.986902 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.740108 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.019s	sys 0.020s
W20260709 13:03:22.988503 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.019s	sys 0.020s
I20260709 13:03:22.987525 14984 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 3, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
W20260709 13:03:22.988909 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 867619us
I20260709 13:03:22.866219 17496 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:22.942584 17277 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:22.925837 15087 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:22.926296 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.926453 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:22.926213 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 253594us
I20260709 13:03:22.826571 17506 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.949672 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:22.943145 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:22.926155 17418 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.326s	user 0.010s	sys 0.003s
I20260709 13:03:22.953909 16946 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:22.954730 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.953860 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 485) took 1834 ms. Trace:
I20260709 13:03:22.961321 15159 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:22.925247 17117 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:22.962553 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:22.925530 17592 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:22.951666 17575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.003s	sys 0.016s
I20260709 13:03:22.933418 17597 consensus_queue.cc:1243] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:22.984894 15163 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:22.823949 17613 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.862118 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:22.910462 17498 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:22.915473 17590 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:22.910895 17601 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:22.924743 17605 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:23.452333 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.454756 15079 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:23.011710 15082 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
W20260709 13:03:23.322293 15237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.016s	sys 0.026s
W20260709 13:03:23.455976 15237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.018s	sys 0.027s
I20260709 13:03:23.004107 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:23.348291 17546 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
W20260709 13:03:23.449905 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 469691us
W20260709 13:03:23.454203 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 504608us
I20260709 13:03:23.459522 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:23.461109 15160 rpcz_store.cc:276] 0709 13:03:21.119179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.119355 (+   176us) service_pool.cc:224] Handling call
0709 13:03:21.119778 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.401659 (+281881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.401688 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.436920 (+ 35232us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.183342 (+746422us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:22.183397 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.183496 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.953608 (+770112us) raft_consensus.cc:1685] finished
0709 13:03:22.953633 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.953787 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":561337,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":14169,"thread_start_us":655207,"threads_started":1,"wal-append.queue_time_us":978753}
I20260709 13:03:23.015036 14984 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.005370 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.156919 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:23.444392 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 415877us
I20260709 13:03:23.444990 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.468080 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:23.084113 17486 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.000s
W20260709 13:03:23.476779 15126 connection.cc:256] RPC call timeout handler was delayed by -1.1803715230000762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
I20260709 13:03:23.478165 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.533566 15848 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:23.501127 16946 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:03:23.504179 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.536788 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.557740 15845 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.003s
I20260709 13:03:23.053761 17622 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:23.004314 17490 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.574167 17622 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.572392 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1648) took 1833 ms. Trace:
I20260709 13:03:23.555292 15161 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:23.576483 15126 connection.cc:256] RPC call timeout handler was delayed by -1.1526669580000544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 13:03:23.576737 15080 rpcz_store.cc:276] 0709 13:03:21.738703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.741049 (+  2346us) service_pool.cc:224] Handling call
0709 13:03:22.692909 (+951860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.907443 (+214534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.572121 (+664678us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400306950. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:23.572317 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151211264}
I20260709 13:03:23.488544 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.579598 15163 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:23.580741 15845 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:03:23.141038 17609 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.003s
I20260709 13:03:23.588788 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:23.590123 15236 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:23.590394 15229 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:23.591463 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:23.272270 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 282045us
I20260709 13:03:23.581722 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 762) took 1246 ms. Trace:
W20260709 13:03:23.029259 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 46104us
I20260709 13:03:23.466632 14984 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:23.540427 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:23.598831 15845 rpcz_store.cc:276] 0709 13:03:22.335195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.397053 (+ 61858us) service_pool.cc:224] Handling call
0709 13:03:22.452402 (+ 55349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.581384 (+1128982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.581645 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.002734 17507 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:23.443637 17285 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.388717 17491 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.457585 17546 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:23.386848 17548 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.638171 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:23.638949 15079 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:23.639431 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:23.640056 15086 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:23.640955 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:23.641799 15848 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:23.114406 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:22.957417 17477 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.579695 17277 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:22.958827 17578 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:23.464197 17522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.003s
I20260709 13:03:23.634320 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:23.632601 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:23.660032 15080 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:03:23.053581 17338 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.578s
I20260709 13:03:23.458853 17431 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:23.664114 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:23.665238 15848 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:23.665657 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:23.631559 15235 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:23.637203 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:23.586033 17429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.018s	sys 0.047s
I20260709 13:03:23.641937 15161 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:23.642234 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.624388 14984 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.666925 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:23.667920 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.668421 14984 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.449316 17504 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:23.468088 17117 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.084403 17298 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.531s
I20260709 13:03:23.644815 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1801) took 1213 ms. Trace:
I20260709 13:03:23.645923 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1660) took 1046 ms. Trace:
I20260709 13:03:23.691869 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:23.676069 17429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.018s	sys 0.047s
W20260709 13:03:23.709013 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74814.6us
I20260709 13:03:23.710180 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:23.711680 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:23.668766 17636 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.546167 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:23.650909 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50372.9us
I20260709 13:03:23.475656 17613 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.475754 17637 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:23.670897 14984 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.635519 17546 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:23.684795 15233 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:03:23.649464 17178 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.662164 17522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.009s	sys 0.007s
I20260709 13:03:23.632655 17486 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:23.676267 15839 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:23.615940 17609 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:23.737931 17656 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.487s	user 0.001s	sys 0.001s
I20260709 13:03:23.690469 15079 rpcz_store.cc:276] 0709 13:03:22.600374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.609414 (+1009040us) service_pool.cc:224] Handling call
0709 13:03:23.645823 (+ 36409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.695564 15080 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260709 13:03:23.647377 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.016s	sys 0.011s
I20260709 13:03:23.694567 15236 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:23.802182 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.803186 15236 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:23.690341 15162 rpcz_store.cc:276] 0709 13:03:22.431220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.462025 (+ 30805us) service_pool.cc:224] Handling call
0709 13:03:23.644625 (+1182600us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b540017dc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:23.644761 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011890560}
W20260709 13:03:23.804045 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.017s	sys 0.011s
W20260709 13:03:23.804493 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 719717us
I20260709 13:03:23.697311 15845 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:23.698261 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.807585 15229 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
W20260709 13:03:23.699851 17655 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.384s	user 0.000s	sys 0.000s
W20260709 13:03:23.700531 17661 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:03:23.665616 17607 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:23.831058 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2261) took 1257 ms. Trace:
I20260709 13:03:23.831578 15088 rpcz_store.cc:276] 0709 13:03:22.573941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.574182 (+   241us) service_pool.cc:224] Handling call
0709 13:03:22.664237 (+ 90055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.664623 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.664655 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.665174 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.443972 (+778798us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:23.444016 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.444095 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.830799 (+386704us) raft_consensus.cc:1685] finished
0709 13:03:23.830824 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.830987 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14639,"thread_start_us":425501,"threads_started":1,"wal-append.queue_time_us":1126365}
I20260709 13:03:23.794162 17651 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.682245 17653 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.526s	user 0.000s	sys 0.001s
I20260709 13:03:23.833966 17651 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.754982 17178 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.835798 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 759) took 1507 ms. Trace:
I20260709 13:03:23.758903 17648 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:23.724651 15086 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:23.836859 17178 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.837157 17648 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:23.726711 17619 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.005s	sys 0.009s
I20260709 13:03:23.837673 17178 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:23.837882 17619 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.005s	sys 0.009s
W20260709 13:03:23.689841 17657 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 13:03:23.838078 17648 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.839437 17648 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:23.840049 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 505) took 1626 ms. Trace:
I20260709 13:03:23.815299 17659 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:23.840550 15232 rpcz_store.cc:276] 0709 13:03:22.213511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.384911 (+171400us) service_pool.cc:224] Handling call
0709 13:03:22.385320 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.385749 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.385794 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.386393 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.912432 (+526039us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:22.912493 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.912747 (+   254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.839781 (+927034us) raft_consensus.cc:1685] finished
0709 13:03:23.839809 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.839971 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":930,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":30712,"thread_start_us":455385,"threads_started":1,"wal-append.queue_time_us":827595}
I20260709 13:03:23.843258 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.844012 15232 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:23.844503 17314 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.024s	user 0.007s	sys 0.000s
I20260709 13:03:23.682941 17622 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:23.845827 17477 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.691622 17117 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:23.742118 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.846165 17506 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:23.846766 15233 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:23.846460 17477 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:23.847361 17477 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.847776 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:23.848388 15233 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:23.722677 17652 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.563s	user 0.001s	sys 0.001s
I20260709 13:03:23.663029 17277 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:23.849406 17605 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:23.849686 17298 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:23.850051 17298 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:23.851042 17277 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.696334 17431 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.712915 17490 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.850893 17298 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.851861 17431 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:23.847918 17546 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.852538 17298 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:23.798063 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 338588us
W20260709 13:03:23.795751 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84155.6us
I20260709 13:03:23.686213 17658 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
W20260709 13:03:23.812898 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 80257.3us
I20260709 13:03:23.876178 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:23.813850 15080 pending_rounds.cc:85] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 2
I20260709 13:03:23.819893 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.879220 15080 pending_rounds.cc:107] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:03:23.820680 17498 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:23.808771 17637 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:23.828626 17665 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:23.829107 17668 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:23.833439 17590 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.486641 17496 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.761937 17578 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.688630 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:23.894333 15848 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:23.760210 15843 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:23.648407 15163 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:23.895838 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1802) took 1464 ms. Trace:
I20260709 13:03:23.855050 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1793) took 1426 ms. Trace:
W20260709 13:03:23.896202 15844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 758) took 1569 ms (client timeout 1887 ms). Trace:
I20260709 13:03:23.854758 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1797) took 1424 ms. Trace:
I20260709 13:03:23.896325 15155 rpcz_store.cc:276] 0709 13:03:22.428598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.434448 (+  5850us) service_pool.cc:224] Handling call
0709 13:03:22.434943 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.435263 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.435304 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.435962 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.889812 (+453850us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:22.889872 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.889971 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.854874 (+964903us) raft_consensus.cc:1685] finished
0709 13:03:23.854890 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.855004 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":453391,"threads_started":1,"wal-append.queue_time_us":613494}
I20260709 13:03:23.897234 15154 rpcz_store.cc:276] 0709 13:03:22.430063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.602759 (+172696us) service_pool.cc:224] Handling call
0709 13:03:22.603157 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.673080 (+ 69923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.673113 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.793657 (+120544us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.243316 (+449659us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:23.243387 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.243485 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.854561 (+611076us) raft_consensus.cc:1685] finished
0709 13:03:23.854581 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.854704 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62041,"raft.queue_time_us":63117,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":10884,"thread_start_us":353350,"threads_started":1,"wal-append.queue_time_us":868740}
I20260709 13:03:23.896198 15163 rpcz_store.cc:276] 0709 13:03:22.431495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.647759 (+1216264us) service_pool.cc:224] Handling call
0709 13:03:23.648175 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.895589 (+247414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.895783 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.862329 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:23.869831 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:23.901181 15232 raft_consensus.cc:2376] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3. Re-sending same reply.
W20260709 13:03:23.896878 15844 rpcz_store.cc:269] 0709 13:03:22.326936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.328300 (+  1364us) service_pool.cc:224] Handling call
0709 13:03:23.895969 (+1567669us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:23.896132 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018664704}
I20260709 13:03:23.816234 17484 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:23.692736 17592 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.879660 15080 raft_consensus.cc:2889] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.734658 17636 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:23.886330 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:23.886897 17431 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:23.836279 15840 rpcz_store.cc:276] 0709 13:03:22.328747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.333405 (+  4658us) service_pool.cc:224] Handling call
0709 13:03:22.367587 (+ 34182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.368107 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.368159 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.368832 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.792105 (+423273us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:22.792148 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.792230 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.835514 (+1043284us) raft_consensus.cc:1685] finished
0709 13:03:23.835551 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.835724 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1670,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":176,"thread_start_us":422698,"threads_started":1,"wal-append.queue_time_us":594495}
I20260709 13:03:23.912256 17671 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:03:23.883888 17522 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:03:23.918380 17496 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:23.852789 17485 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.736s	user 0.002s	sys 0.009s
I20260709 13:03:23.859647 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:23.921133 15086 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:23.853056 17546 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:23.927697 16946 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:23.928613 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.929788 15155 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:03:23.892390 15203 connection.cc:256] RPC call timeout handler was delayed by -1.3958018450000509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 13:03:23.900279 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.905679 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:23.944162 15163 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:23.906050 15233 raft_consensus.cc:2410] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:23.897728 17553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.003s	sys 0.015s
I20260709 13:03:23.838099 17178 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:23.855973 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:23.866199 17314 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:23.987463 15203 connection.cc:256] RPC call timeout handler was delayed by -1.1456018449999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 13:03:23.987958 15203 connection.cc:256] RPC call timeout handler was delayed by -1.0529018449999512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 13:03:23.988372 15203 connection.cc:256] RPC call timeout handler was delayed by -1.0331018450000329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 13:03:23.909680 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:23.884249 17598 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.059s	user 0.005s	sys 0.006s
I20260709 13:03:23.912513 15083 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:23.910614 17277 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:23.884465 17597 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:03:23.741740 14984 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 3, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 3 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.881824 17622 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:23.904862 17443 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.078s	user 0.007s	sys 0.004s
I20260709 13:03:23.927697 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:23.947104 17553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.003s	sys 0.015s
I20260709 13:03:24.011181 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:23.945423 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:23.942337 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1661) took 1340 ms. Trace:
I20260709 13:03:23.947081 16946 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:24.017530 15163 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:23.903105 15232 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:03:23.847792 17477 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:23.859395 17429 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.942214 17485 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:23.904330 17498 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.995748 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.052130 15228 raft_consensus.cc:2410] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:23.913499 17506 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:03:24.000100 15157 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:24.002095 14984 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:23.987279 17178 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:24.004695 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.017266 15080 rpcz_store.cc:276] 0709 13:03:22.602657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.673917 (+1071260us) service_pool.cc:224] Handling call
0709 13:03:23.695135 (+ 21218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.942016 (+246881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.942262 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:24.018648 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1784) took 1158 ms. Trace:
I20260709 13:03:24.053369 14984 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 2 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:24.054209 15232 rpcz_store.cc:276] 0709 13:03:22.859881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.902298 (+1042417us) service_pool.cc:224] Handling call
0709 13:03:23.902761 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.018337 (+115576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.018580 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.946266 17592 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:23.993013 17590 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.900342 17672 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:24.063760 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 511) took 1833 ms. Trace:
I20260709 13:03:24.064289 15237 rpcz_store.cc:276] 0709 13:03:22.230238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.576104 (+345866us) service_pool.cc:224] Handling call
0709 13:03:22.668662 (+ 92558us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:22.693710 (+ 25048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.693742 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:22.719911 (+ 26169us) log.cc:844] Serialized 52 byte log entry
0709 13:03:23.456355 (+736444us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:23.456396 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.456462 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.063459 (+606997us) raft_consensus.cc:1685] finished
0709 13:03:24.063489 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.063685 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11300,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":25814,"thread_start_us":691046,"threads_started":1,"wal-append.queue_time_us":910523}
I20260709 13:03:24.021380 15229 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:24.067837 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1785) took 1207 ms. Trace:
I20260709 13:03:24.068269 15229 rpcz_store.cc:276] 0709 13:03:22.860111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.987337 (+1127226us) service_pool.cc:224] Handling call
0709 13:03:24.021141 (+ 33804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.067473 (+ 46332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.067765 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:24.069574 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1792) took 1164 ms. Trace:
I20260709 13:03:24.070076 15228 rpcz_store.cc:276] 0709 13:03:22.905276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.995458 (+1090182us) service_pool.cc:224] Handling call
0709 13:03:24.069485 (+ 74027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.999330 17496 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:24.024255 15154 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:23.999637 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:24.058099 15085 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:24.059155 14984 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 4, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:23.942209 17484 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:03:24.020545 17598 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:24.131595 17598 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:24.024277 17606 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.018s	sys 0.016s
I20260709 13:03:24.128360 14984 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:24.023869 17277 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:24.132017 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.139529 15154 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:24.134073 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.139262 14984 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:24.141402 14984 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 3, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 3 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:24.025406 15156 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:24.143890 14984 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:24.061996 17668 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.147583 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.052767 17431 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:24.022257 15086 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:24.019867 17622 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.143916 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.150732 14984 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:24.176473 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.176680 15232 raft_consensus.cc:2376] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4. Re-sending same reply.
I20260709 13:03:24.177135 15237 raft_consensus.cc:2376] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
I20260709 13:03:24.177568 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.178133 15232 raft_consensus.cc:2376] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
I20260709 13:03:24.178254 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.178779 15229 raft_consensus.cc:2376] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
I20260709 13:03:24.178982 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:24.144359 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 266749us
I20260709 13:03:24.172209 15228 raft_consensus.cc:2376] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5. Re-sending same reply.
I20260709 13:03:24.180285 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.181317 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.133370 17484 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:24.153276 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:24.184405 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:24.152429 17606 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.021s	sys 0.020s
I20260709 13:03:24.210153 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.230491 15155 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:24.210512 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.211293 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.057497 17429 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.212543 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.241588 15232 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:24.242859 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.215199 17613 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.223062 17622 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:24.225804 17683 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:24.228174 17477 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:24.228760 17484 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:24.231762 15228 raft_consensus.cc:2376] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5. Re-sending same reply.
W20260709 13:03:24.232394 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48083.1us
I20260709 13:03:24.268308 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.269905 15232 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:24.271147 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.185606 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:24.062872 17676 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:03:24.237701 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.238922 15237 raft_consensus.cc:2410] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:24.258030 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 104874us
I20260709 13:03:24.276906 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:24.260757 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.104789 17590 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.066838 17498 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.173030 17607 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:24.277976 15229 raft_consensus.cc:2376] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4. Re-sending same reply.
W20260709 13:03:24.236552 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 744) took 5325 ms (client timeout 2694 ms). Trace:
I20260709 13:03:24.283531 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.300505 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.174759 17431 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:24.129354 17496 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.287326 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.302341 15228 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:03:24.289681 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 104144us
W20260709 13:03:24.305742 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.290669 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:24.310712 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.312270 15155 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:03:24.120782 17672 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.318125 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.059897 17637 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.259464 17677 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:24.331537 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.337970 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:24.266968 17605 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:24.339193 15229 raft_consensus.cc:2376] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
W20260709 13:03:24.297931 15839 rpcz_store.cc:269] 0709 13:03:18.911128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.000796 (+ 89668us) service_pool.cc:224] Handling call
0709 13:03:20.961948 (+1961152us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:24.236249 (+3274301us) spinlock_profiling.cc:243] Waited 2.62 s on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.236412 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6022264320}
I20260709 13:03:24.312057 17622 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:24.328204 17606 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:24.341709 17484 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.345285 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.331568 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:24.347960 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1779) took 1489 ms. Trace:
I20260709 13:03:24.348026 17496 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.349795 17498 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.351552 17429 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.313848 17676 consensus_queue.cc:1243] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:24.418269 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:24.419126 17631 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.370s	user 0.009s	sys 0.003s
I20260709 13:03:24.419225 15842 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:24.420445 15158 raft_consensus.cc:2410] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:24.355479 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 512) took 2111 ms. Trace:
I20260709 13:03:24.421078 15231 rpcz_store.cc:276] 0709 13:03:22.244088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.745896 (+501808us) service_pool.cc:224] Handling call
0709 13:03:22.822750 (+ 76854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.947629 (+124879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.355193 (+1407564us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b540005cb50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.355395 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74063744}
W20260709 13:03:24.422382 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2363) took 1530 ms (client timeout 2012 ms). Trace:
I20260709 13:03:24.423281 15231 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:24.423758 17314 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.424397 17314 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.391624 17677 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:24.426234 17677 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:24.311866 17607 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:24.357553 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2367) took 1465 ms (client timeout 1609 ms). Trace:
W20260709 13:03:24.451350 15161 rpcz_store.cc:269] 0709 13:03:22.892528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.923871 (+ 31343us) service_pool.cc:224] Handling call
0709 13:03:24.357340 (+1433469us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.357480 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232366848}
W20260709 13:03:24.360260 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1778) took 1501 ms (client timeout 1747 ms). Trace:
W20260709 13:03:24.452723 15235 rpcz_store.cc:269] 0709 13:03:22.858318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.859307 (+   989us) service_pool.cc:224] Handling call
0709 13:03:24.360024 (+1500717us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54000abed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.360175 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418545024}
W20260709 13:03:24.361847 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1808) took 1689 ms (client timeout 1623 ms). Trace:
I20260709 13:03:24.180143 17668 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.364312 17590 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.365450 17431 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:24.368383 17672 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.369279 17611 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.339005 17277 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.133062 17598 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:24.373679 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1662) took 1722 ms (client timeout 2103 ms). Trace:
I20260709 13:03:24.376825 17603 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.377487 17430 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:24.378545 17507 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:24.378849 17660 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:24.386271 17443 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:24.387293 17490 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.389638 17636 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:24.390909 17651 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:24.392282 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.393074 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:24.393998 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.395437 15237 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 3: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:24.399677 15155 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:24.345745 17675 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:24.399420 15230 rpcz_store.cc:276] 0709 13:03:22.858563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.674352 (+815789us) service_pool.cc:224] Handling call
0709 13:03:23.690767 (+ 16415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.691032 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.691096 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.721879 (+ 30783us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.167405 (+445526us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:24.167462 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.167564 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.347727 (+180163us) raft_consensus.cc:1685] finished
0709 13:03:24.347750 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.347893 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":406176,"threads_started":1,"wal-append.queue_time_us":508489}
I20260709 13:03:24.400942 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:24.405594 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:24.406231 17622 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:24.402467 17606 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:24.422948 15158 rpcz_store.cc:269] 0709 13:03:22.891555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.049332 (+157777us) service_pool.cc:224] Handling call
0709 13:03:24.422107 (+1372775us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540033ed54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.422303 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174598400}
I20260709 13:03:24.312462 17619 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.364400 17638 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:24.530174 15231 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:24.338203 17117 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:24.681155 15848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.038s	sys 0.031s
I20260709 13:03:24.833155 17606 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:24.835515 17117 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.836654 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:24.660470 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.050s	sys 0.152s
I20260709 13:03:24.837324 15158 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:24.837774 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.061s	sys 0.164s
I20260709 13:03:24.839035 15229 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:03:24.838210 17430 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:24.712551 15842 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
W20260709 13:03:24.840381 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 755) took 1488 ms (client timeout 1803 ms). Trace:
I20260709 13:03:24.839715 17622 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:24.672691 17669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.016s	sys 0.051s
W20260709 13:03:24.841820 17669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.016s	sys 0.052s
I20260709 13:03:24.842552 17490 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:24.843027 17507 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:24.844797 17490 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:24.559924 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:24.833658 15848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.046s	sys 0.043s
I20260709 13:03:24.849025 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:24.750806 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.749398 17659 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.908s	user 0.008s	sys 0.003s
I20260709 13:03:24.833355 17638 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:24.838440 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 969076us
W20260709 13:03:24.810545 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.852309 15232 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:24.850872 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.909339 15155 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:03:24.857336 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 464279us
I20260709 13:03:24.930375 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:24.859308 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 458363us
I20260709 13:03:24.864152 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:24.840823 15842 rpcz_store.cc:269] 0709 13:03:23.352532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.457611 (+105079us) service_pool.cc:224] Handling call
0709 13:03:24.418038 (+960427us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:24.840310 (+422272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40531840}
I20260709 13:03:24.838214 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:24.878383 15161 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:24.950554 15158 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:24.950870 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1807) took 2278 ms. Trace:
I20260709 13:03:24.951469 15161 rpcz_store.cc:276] 0709 13:03:22.672367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.850498 (+2178131us) service_pool.cc:224] Handling call
0709 13:03:24.878083 (+ 27585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.950530 (+ 72447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.950804 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:24.952996 15126 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.539s (SENT)
I20260709 13:03:24.953683 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:24.869676 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.955060 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.860404 15083 rpcz_store.cc:269] 0709 13:03:22.651880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.832878 (+180998us) service_pool.cc:224] Handling call
0709 13:03:24.373454 (+1540576us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b54002378d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.373600 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203068288}
I20260709 13:03:24.998353 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:25.001809 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:24.767725 17178 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.002s
W20260709 13:03:24.982738 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52443us
I20260709 13:03:24.858099 17631 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:24.980746 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 116653us
I20260709 13:03:25.024524 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:25.026515 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:24.997618 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2380) took 1110 ms. Trace:
I20260709 13:03:24.852778 17659 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:24.854717 17431 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.007790 17669 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:25.035686 17117 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:24.846068 17507 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.005877 17675 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.036727 17507 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.037084 17675 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.039690 17675 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:24.991174 17611 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:25.002315 17648 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.162s	user 0.013s	sys 0.000s
I20260709 13:03:24.841034 17430 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:25.035317 15228 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.732s	user 0.005s	sys 0.005s
I20260709 13:03:25.061684 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:24.859593 17660 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:25.001775 17603 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:25.001960 17672 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:25.087015 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 752) took 2080 ms. Trace:
I20260709 13:03:25.087566 15845 rpcz_store.cc:276] 0709 13:03:23.006876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.607070 (+600194us) service_pool.cc:224] Handling call
0709 13:03:23.607483 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.807294 (+199811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.086682 (+1279388us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b54006d7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:25.086936 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204551552}
I20260709 13:03:24.855998 17619 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.031790 17668 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.095501 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.049309 17611 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:25.064898 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1663) took 2390 ms (client timeout 1592 ms). Trace:
I20260709 13:03:25.073891 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.105552 15082 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:24.844815 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:25.029457 15155 rpcz_store.cc:276] 0709 13:03:23.887596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.849784 (+962188us) service_pool.cc:224] Handling call
0709 13:03:24.997531 (+147747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.079944 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 506) took 2853 ms. Trace:
I20260709 13:03:25.123845 15234 rpcz_store.cc:276] 0709 13:03:22.213756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.454580 (+240824us) service_pool.cc:224] Handling call
0709 13:03:22.454874 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.049867 (+594993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.049898 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.049939 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:23.049968 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.066807 (+2016839us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b5400294d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:25.066867 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.067043 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367927680}
W20260709 13:03:24.550350 15156 rpcz_store.cc:269] 0709 13:03:22.672618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.824337 (+151719us) service_pool.cc:224] Handling call
0709 13:03:24.361572 (+1537235us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540017f054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:24.361746 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278950144}
I20260709 13:03:25.081099 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1637) took 5466 ms. Trace:
I20260709 13:03:25.092671 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 799) took 2152 ms. Trace:
I20260709 13:03:25.129374 15085 rpcz_store.cc:276] 0709 13:03:19.597296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.963705 (+366409us) service_pool.cc:224] Handling call
0709 13:03:19.964045 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.128159 (+4164114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.063408 (+935249us) spinlock_profiling.cc:243] Waited 4.09 s on lock 0x7b540011a8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:25.063607 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9414877440}
I20260709 13:03:25.048676 17609 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.057922 17675 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.102305 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:24.823843 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:25.111465 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:25.179338 15842 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:25.119128 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92711.5us
W20260709 13:03:25.104297 15087 rpcz_store.cc:269] 0709 13:03:22.674539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.739190 (+ 64651us) service_pool.cc:224] Handling call
0709 13:03:25.064625 (+2325435us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b5400307354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:25.064784 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2580864}
I20260709 13:03:25.020356 17178 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.147181 15127 connection.cc:256] RPC call timeout handler was delayed by -1.1560629110000491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 13:03:25.177532 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:25.181531 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.182648 15127 connection.cc:256] RPC call timeout handler was delayed by -1.0489629109999896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260709 13:03:24.855432 17443 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.099184 17546 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.155s	user 0.007s	sys 0.006s
I20260709 13:03:24.854454 17638 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.078933 17603 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:25.189496 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2368) took 2296 ms (client timeout 1887 ms). Trace:
W20260709 13:03:25.191185 15157 rpcz_store.cc:269] 0709 13:03:22.892820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.926043 (+ 33223us) service_pool.cc:224] Handling call
0709 13:03:25.189095 (+2263052us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54001c70d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:25.189408 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249573248}
I20260709 13:03:25.192169 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1787) took 2331 ms. Trace:
W20260709 13:03:25.193576 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.194496 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.201364 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1788) took 2340 ms. Trace:
I20260709 13:03:25.201952 15232 rpcz_store.cc:276] 0709 13:03:22.863035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.406357 (+1543322us) service_pool.cc:224] Handling call
0709 13:03:24.852071 (+445714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.941255 (+ 89184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.201017 (+259762us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400175f50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:25.201276 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022693632}
W20260709 13:03:25.126382 17592 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.065s	user 0.006s	sys 0.007s
I20260709 13:03:25.192690 15229 rpcz_store.cc:276] 0709 13:03:22.860595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.404244 (+1543649us) service_pool.cc:224] Handling call
0709 13:03:24.556442 (+152198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.839466 (+283024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.191805 (+352339us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54000582d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:25.192093 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649296128}
I20260709 13:03:25.129479 15847 rpcz_store.cc:276] 0709 13:03:22.940616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.629596 (+688980us) service_pool.cc:224] Handling call
0709 13:03:23.637825 (+  8229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.695599 (+ 57774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.695629 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.723498 (+ 27869us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.060059 (+336561us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:24.060100 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.060170 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.092372 (+1032202us) raft_consensus.cc:1685] finished
0709 13:03:25.092400 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.092592 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42055,"raft.queue_time_us":90546,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":11823,"thread_start_us":246277,"threads_started":1,"wal-append.queue_time_us":1021099}
I20260709 13:03:25.122481 17611 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:25.214529 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:25.208010 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:25.210651 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2365) took 2318 ms. Trace:
I20260709 13:03:25.196727 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:25.219019 15155 rpcz_store.cc:276] 0709 13:03:22.892044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.124296 (+2232252us) service_pool.cc:224] Handling call
0709 13:03:25.209942 (+ 85646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.210193 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.210233 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.210298 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:25.210345 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.210426 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.210579 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.220882 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:25.221035 15155 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:03:25.222321 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2378) took 1335 ms. Trace:
I20260709 13:03:25.179139 17430 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.127578 17668 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:25.222831 15155 rpcz_store.cc:276] 0709 13:03:23.887049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.220316 (+1333267us) service_pool.cc:224] Handling call
0709 13:03:25.220735 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.221958 (+  1223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.222245 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.225960 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:25.229926 17592 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:25.223227 17117 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.227267 17546 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:25.231352 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:25.240272 17663 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.489s	user 0.003s	sys 0.010s
I20260709 13:03:25.241040 17684 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:25.206327 17603 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.207921 17609 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:25.185328 15839 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:25.251730 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:25.208761 17636 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:25.254783 17669 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:25.098659 17672 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.204164 15843 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:25.263324 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.274019 17484 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.277209 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.278064 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:25.278316 17601 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.245471 17546 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.190639 17675 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.286944 15845 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
W20260709 13:03:25.296445 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44821.9us
I20260709 13:03:25.296912 17514 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:25.298748 17314 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.245064 17592 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.223398 17547 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.256426 17638 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:25.316195 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:25.320202 17675 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:25.301151 17684 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.301406 17669 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.300758 17178 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.334780 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:25.315481 17546 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.280925 17607 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:25.341544 17669 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:25.333815 17592 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.339444 17547 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.358284 17677 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.931s	user 0.004s	sys 0.009s
I20260709 13:03:25.359155 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:25.318303 17514 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:25.362115 17668 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:25.319043 17601 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.302135 17636 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.284173 17609 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.363121 17431 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:25.363485 17314 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:25.365031 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 498) took 2407 ms. Trace:
I20260709 13:03:25.366724 17684 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:25.376771 15079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.062s	sys 0.127s
W20260709 13:03:25.425398 15079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.484s	user 0.062s	sys 0.128s
W20260709 13:03:25.383680 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.378926 17522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.369s	user 0.018s	sys 0.020s
I20260709 13:03:25.412027 17485 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.412751 17486 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.418893 17677 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.409951 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.439611 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.239637 17660 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:25.385000 17418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.327s	user 0.008s	sys 0.039s
W20260709 13:03:25.426136 17504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.002s	sys 0.035s
W20260709 13:03:25.417770 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.355s	user 0.009s	sys 0.032s
I20260709 13:03:25.433739 17485 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.445181 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.009s	sys 0.032s
I20260709 13:03:25.434759 17514 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.435204 17619 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.385648 17178 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:25.430974 17522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.425s	user 0.018s	sys 0.021s
I20260709 13:03:25.436364 17546 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.425331 17668 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:25.429456 15842 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
W20260709 13:03:25.437836 17606 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.603s	user 0.015s	sys 0.000s
I20260709 13:03:25.488746 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:25.508160 15845 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:25.444823 17504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.002s	sys 0.036s
W20260709 13:03:25.442101 17418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.388s	user 0.010s	sys 0.039s
I20260709 13:03:25.521802 15235 raft_consensus.cc:1217] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:03:25.525179 15235 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:25.482224 17660 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.480417 17672 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:25.529855 16946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.507s	user 0.009s	sys 0.036s
I20260709 13:03:25.530035 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:25.530365 16946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.512s	user 0.009s	sys 0.036s
I20260709 13:03:25.530843 16946 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:25.439529 17486 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.527734 15845 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:25.466787 15843 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:25.512374 17514 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:25.527359 15844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.423s	user 0.009s	sys 0.036s
W20260709 13:03:25.554589 15844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.451s	user 0.009s	sys 0.037s
I20260709 13:03:25.522177 17522 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:25.513808 17443 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:25.527513 17609 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.513042 17178 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.514003 17606 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.563407 17117 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:25.529490 17575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.443s	user 0.028s	sys 0.027s
I20260709 13:03:25.598193 17485 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.631376 17636 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:25.632436 17698 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:25.633272 17547 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.639034 17603 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.640038 17601 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:25.640616 17671 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:25.641185 17671 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:25.644670 17677 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:25.645341 17606 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:25.627758 17575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.561s	user 0.028s	sys 0.028s
I20260709 13:03:25.588788 17611 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.647620 17631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.011s
I20260709 13:03:25.590623 17668 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.715533 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.826s	user 0.007s	sys 0.017s
W20260709 13:03:25.716367 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.827s	user 0.007s	sys 0.017s
I20260709 13:03:25.718245 15083 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:03:25.718568 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.698s	user 0.040s	sys 0.047s
W20260709 13:03:25.719254 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.707s	user 0.040s	sys 0.048s
I20260709 13:03:25.719758 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1664) took 3044 ms. Trace:
I20260709 13:03:25.721202 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1800) took 3290 ms. Trace:
I20260709 13:03:25.720477 15079 rpcz_store.cc:276] 0709 13:03:22.674889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.800791 (+1125902us) service_pool.cc:224] Handling call
0709 13:03:23.880405 (+ 79614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.880717 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.880753 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.883437 (+  2684us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.425883 (+1542446us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:25.425924 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.425990 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.718796 (+292806us) raft_consensus.cc:1685] finished
0709 13:03:25.718852 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.719424 (+   572us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1484497,"threads_started":1,"wal-append.queue_time_us":1755069}
I20260709 13:03:25.724268 15082 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:25.721791 15159 rpcz_store.cc:276] 0709 13:03:22.430914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.609777 (+1178863us) service_pool.cc:224] Handling call
0709 13:03:23.610200 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.610493 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:24.012157 (+401664us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.720184 (+1708027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.720313 (+   129us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:25.720541 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.720695 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.720998 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":819693,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"thread_start_us":2076235,"threads_started":2}
W20260709 13:03:25.726718 15154 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.016s
W20260709 13:03:25.727236 15154 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.016s
I20260709 13:03:25.730836 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:25.731602 15085 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:25.732481 15084 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:25.733506 15162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.707s	user 0.005s	sys 0.022s
W20260709 13:03:25.734109 15162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.708s	user 0.005s	sys 0.022s
I20260709 13:03:25.734195 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1786) took 2873 ms. Trace:
W20260709 13:03:25.737334 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.871s	user 0.008s	sys 0.080s
I20260709 13:03:25.737661 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1782) took 2878 ms. Trace:
W20260709 13:03:25.738037 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.871s	user 0.008s	sys 0.080s
I20260709 13:03:25.738268 15236 rpcz_store.cc:276] 0709 13:03:22.859310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.811229 (+951919us) service_pool.cc:224] Handling call
0709 13:03:23.811638 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.811918 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.811957 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.812631 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.253750 (+441119us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:24.253790 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.253858 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.737130 (+1483272us) raft_consensus.cc:1685] finished
0709 13:03:25.737164 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.737463 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":352416,"threads_started":1,"wal-append.queue_time_us":1651524}
I20260709 13:03:25.423430 15156 rpcz_store.cc:276] 0709 13:03:22.957834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.191143 (+2233309us) service_pool.cc:224] Handling call
0709 13:03:25.282526 (+ 91383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.315674 (+ 33148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.318126 (+  2452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.334074 (+ 15948us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:25.337295 (+  3221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.341931 (+  4636us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.364953 (+ 23022us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.734752 15233 rpcz_store.cc:276] 0709 13:03:22.860347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.995544 (+1135197us) service_pool.cc:224] Handling call
0709 13:03:23.995953 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.996402 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.996442 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.996973 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.445618 (+1448645us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:25.482507 (+ 36889us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.482806 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.482912 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.639230 (+156318us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b180042c640. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:25.642755 (+  3525us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b180042c640. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:25.733431 (+ 90676us) raft_consensus.cc:1685] finished
0709 13:03:25.733487 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.733824 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59508,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":25902,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":22003,"spinlock_wait_cycles":15449600,"thread_start_us":1382801,"threads_started":1,"wal-append.queue_time_us":1631124}
W20260709 13:03:25.745849 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.867s	user 0.003s	sys 0.025s
I20260709 13:03:25.745983 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2362) took 2837 ms. Trace:
W20260709 13:03:25.746191 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.746520 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.868s	user 0.004s	sys 0.025s
W20260709 13:03:25.747027 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.86819e+06us
W20260709 13:03:25.748270 15841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.356s	user 0.005s	sys 0.009s
W20260709 13:03:25.748682 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.748912 15841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.357s	user 0.005s	sys 0.010s
I20260709 13:03:25.750046 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 759) took 2059 ms. Trace:
I20260709 13:03:25.746541 15154 rpcz_store.cc:276] 0709 13:03:22.891260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.043905 (+2152645us) service_pool.cc:224] Handling call
0709 13:03:25.085010 (+ 41105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.728147 (+643137us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:25.728204 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.728308 (+   104us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:25.728387 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.728513 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.728715 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":397809,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":8703,"thread_start_us":642205,"threads_started":1}
I20260709 13:03:25.748613 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 805) took 1359 ms. Trace:
I20260709 13:03:25.750916 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 804) took 1366 ms. Trace:
I20260709 13:03:25.751145 15843 rpcz_store.cc:276] 0709 13:03:24.388524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.398563 (+ 10039us) service_pool.cc:224] Handling call
0709 13:03:25.748247 (+1349684us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.752362 17314 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:25.752581 17430 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:25.753749 17314 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.753998 17430 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.757443 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.922508 17430 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.750612 15844 rpcz_store.cc:276] 0709 13:03:23.690293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.909927 (+219634us) service_pool.cc:224] Handling call
0709 13:03:23.947098 (+ 37171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.947465 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.947508 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.006651 (+ 59143us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.555076 (+1548425us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:25.555133 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.555220 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.749283 (+194063us) raft_consensus.cc:1685] finished
0709 13:03:25.749330 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.749757 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13347,"raft.queue_time_us":223485,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":58088,"thread_start_us":1450963,"threads_started":1,"wal-append.queue_time_us":1614338}
I20260709 13:03:25.745342 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.922621 17546 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.877072 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:25.658108 17631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.011s
I20260709 13:03:25.922505 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:25.745846 15156 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:03:25.921252 17496 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.007s	sys 0.013s
I20260709 13:03:25.917706 17603 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.752471 15847 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:25.656795 17671 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.949110 17631 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:25.949891 17611 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.950067 17671 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:25.953800 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.955169 15088 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:25.955983 15080 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:25.956177 17609 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.956508 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:25.961020 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.961508 15088 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:25.961855 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.681684 17669 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.963869 15088 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:25.965570 17669 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.966419 16946 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:25.946228 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 189781us
I20260709 13:03:25.968986 17601 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:25.970134 17601 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:25.971926 15087 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:25.972996 17660 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.974077 17660 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:25.921250 17703 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.995222 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:25.996309 15086 raft_consensus.cc:2410] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:25.999745 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.005048 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.006511 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1703) took 1863 ms. Trace:
I20260709 13:03:26.007067 15235 rpcz_store.cc:276] 0709 13:03:24.142818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.453715 (+310897us) service_pool.cc:224] Handling call
0709 13:03:24.812625 (+358910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.525633 (+713008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.005893 (+480260us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540005bc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:26.006277 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630586240}
W20260709 13:03:26.008924 17496 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.602s	user 0.008s	sys 0.013s
I20260709 13:03:25.970947 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1803) took 3539 ms. Trace:
I20260709 13:03:26.000653 15084 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:25.662781 17658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.016s	sys 0.018s
I20260709 13:03:25.974125 17178 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:26.018333 17486 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:26.018620 17660 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:25.751573 15841 rpcz_store.cc:276] 0709 13:03:24.388135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.390391 (+  2256us) service_pool.cc:224] Handling call
0709 13:03:24.390709 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.391030 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:24.391442 (+   412us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.750095 (+1358653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.750235 (+   140us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:25.750386 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.750567 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.750838 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":648,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":4422,"raft.queue_time_us":173390,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":309,"thread_start_us":1357884,"threads_started":1}
I20260709 13:03:25.982082 17609 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.032878 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2382) took 1639 ms. Trace:
I20260709 13:03:26.033527 15156 rpcz_store.cc:276] 0709 13:03:24.372509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.742024 (+1369515us) service_pool.cc:224] Handling call
0709 13:03:25.744947 (+  2923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.944977 (+200030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.011392 (+ 66415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.053766 15079 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:26.055805 17314 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:25.951403 17631 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.010324 15162 rpcz_store.cc:276] 0709 13:03:22.431754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.833053 (+1401299us) service_pool.cc:224] Handling call
0709 13:03:23.833371 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.855790 (+ 22419us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:24.025301 (+169511us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.734946 (+1709645us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.735102 (+   156us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:25.735208 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.970593 (+235385us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400287dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:25.970663 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.970863 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":574746,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":223296,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"spinlock_wait_cycles":51020160,"thread_start_us":1849324,"threads_started":2}
W20260709 13:03:26.092165 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.057595 15202 connection.cc:256] RPC call timeout handler was delayed by -1.3951992460000611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 13:03:26.121714 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 151923us
W20260709 13:03:26.105393 15202 connection.cc:256] RPC call timeout handler was delayed by -1.1503992460000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:03:26.131713 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.141160 17117 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:26.141624 17117 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:26.142894 17117 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.145648 15088 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
W20260709 13:03:26.121089 17658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.028s	sys 0.039s
W20260709 13:03:26.148152 15054 connection.cc:256] RPC call timeout handler was delayed by -1.96157388000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:03:26.136194 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2266) took 1041 ms. Trace:
I20260709 13:03:26.121106 17619 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:26.151507 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.151564 17619 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.153105 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:26.154520 17619 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:26.154922 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.155120 17484 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.845s	user 0.016s	sys 0.003s
I20260709 13:03:26.157840 17619 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.135641 17601 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:26.150068 15083 rpcz_store.cc:276] 0709 13:03:25.095078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.095215 (+   137us) service_pool.cc:224] Handling call
0709 13:03:26.098542 (+1003327us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b540014d554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:26.136114 (+ 37572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427226880}
I20260709 13:03:26.087942 17703 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:26.179204 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.164098 15156 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:26.180164 17601 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.213835 15162 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:26.218182 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 512) took 1018 ms. Trace:
I20260709 13:03:26.218942 15162 rpcz_store.cc:276] 0709 13:03:25.196507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.136529 (+940022us) service_pool.cc:224] Handling call
0709 13:03:26.174740 (+ 38211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.214449 (+ 39709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.214715 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.215622 15087 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:26.218158 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47294.8us
I20260709 13:03:26.225144 15086 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:03:26.208854 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 511) took 1007 ms. Trace:
W20260709 13:03:26.251010 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.254925 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.216s	user 0.007s	sys 0.025s
W20260709 13:03:26.254766 15052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.008s	sys 0.023s
I20260709 13:03:26.212615 17619 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.241729 17277 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.013s	sys 0.018s
W20260709 13:03:26.259979 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.255s	user 0.007s	sys 0.025s
W20260709 13:03:26.260066 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.009s	sys 0.023s
I20260709 13:03:26.275682 15846 raft_consensus.cc:2410] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:03:26.275876 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 2.36606e+06us
I20260709 13:03:26.276338 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:26.278414 15128 connection.cc:256] RPC call timeout handler was delayed by -2.4172533829998883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 13:03:26.225813 17601 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.265125 15848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.338s	user 0.021s	sys 0.009s
I20260709 13:03:26.279475 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:26.265523 17277 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.013s	sys 0.019s
W20260709 13:03:26.288815 15848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.022s	sys 0.009s
W20260709 13:03:26.265228 15840 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.002s	sys 0.020s
I20260709 13:03:26.279755 17486 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.292506 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 956405us
I20260709 13:03:26.238539 15156 rpcz_store.cc:276] 0709 13:03:25.195736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.035089 (+839353us) service_pool.cc:224] Handling call
0709 13:03:26.035715 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.182984 (+147269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.200364 (+ 17380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:26.288149 17598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.146s	user 0.004s	sys 0.027s
W20260709 13:03:26.323550 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.023s	sys 0.062s
W20260709 13:03:26.322369 15128 connection.cc:256] RPC call timeout handler was delayed by -2.0235674639999388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 13:03:26.284372 17637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.013s	sys 0.018s
W20260709 13:03:26.292533 17695 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.008s	sys 0.016s
W20260709 13:03:26.327523 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.023s	sys 0.062s
W20260709 13:03:26.314030 17638 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.861s	user 0.012s	sys 0.010s
W20260709 13:03:26.318500 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.018s	sys 0.013s
W20260709 13:03:26.326047 15081 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.009s	sys 0.022s
I20260709 13:03:26.292539 17727 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.322189 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.008s	sys 0.022s
W20260709 13:03:26.305338 17622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.038s	sys 0.115s
I20260709 13:03:26.329706 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:26.329151 15813 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.003s	sys 0.018s
W20260709 13:03:26.316941 15840 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.003s	sys 0.021s
W20260709 13:03:26.322561 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.257s	user 0.016s	sys 0.014s
W20260709 13:03:26.325382 14984 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.026s	sys 0.030s
W20260709 13:03:26.334882 17598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.005s	sys 0.027s
W20260709 13:03:26.349792 15813 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.004s	sys 0.018s
W20260709 13:03:26.351987 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.397s	user 0.017s	sys 0.014s
W20260709 13:03:26.349900 15081 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.278s	user 0.009s	sys 0.023s
W20260709 13:03:26.350322 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.019s	sys 0.013s
W20260709 13:03:26.353006 14984 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.116s	user 0.027s	sys 0.030s
W20260709 13:03:26.352034 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.009s	sys 0.022s
W20260709 13:03:26.341182 17637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.013s	sys 0.019s
W20260709 13:03:26.351979 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.010s	sys 0.023s
I20260709 13:03:26.355172 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:26.364439 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:26.378659 15163 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 13:03:26.347507 17695 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.009s	sys 0.017s
W20260709 13:03:26.384738 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.04993e+06us
W20260709 13:03:26.386603 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.43353e+06us
W20260709 13:03:26.387240 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.402794 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.387342 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 848460us
I20260709 13:03:26.388401 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 494) took 3566 ms. Trace:
W20260709 13:03:26.350309 17622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.038s	sys 0.116s
W20260709 13:03:26.405519 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.412087 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.409454 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.011s	sys 0.023s
W20260709 13:03:26.395440 15201 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.006s	sys 0.020s
W20260709 13:03:26.395514 15161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.225s	user 0.000s	sys 0.026s
I20260709 13:03:26.402190 15163 pending_rounds.cc:85] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 2
W20260709 13:03:26.418592 15201 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.607s	user 0.006s	sys 0.020s
I20260709 13:03:26.408702 17710 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:26.420497 15087 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:26.420599 15161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.414s	user 0.001s	sys 0.027s
I20260709 13:03:26.417264 15163 pending_rounds.cc:107] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:03:26.403890 15158 rpcz_store.cc:276] 0709 13:03:22.821708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.951527 (+2129819us) service_pool.cc:224] Handling call
0709 13:03:24.951896 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.952309 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.952351 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.299908 (+347557us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b2c00000448. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:25.300400 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.383363 (+1082963us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:26.383468 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.383575 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.387921 (+  4346us) raft_consensus.cc:1685] finished
0709 13:03:26.387949 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.388300 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":385874,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":1392,"spinlock_wait_cycles":11010304,"thread_start_us":1055833,"threads_started":1,"wal-append.queue_time_us":1083759}
I20260709 13:03:26.431306 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:26.432018 15158 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:26.428419 15163 raft_consensus.cc:2889] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:26.436952 15128 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.592s (SENT) [suppressed 34 similar messages]
I20260709 13:03:26.440925 17619 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:26.425940 15235 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:26.443163 17703 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:26.345464 17486 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.424782 17668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.004s	sys 0.018s
I20260709 13:03:26.420189 17710 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.501408 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:26.455255 17668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.004s	sys 0.019s
W20260709 13:03:26.461112 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.64936e+06us
W20260709 13:03:26.526757 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 89895.9us
I20260709 13:03:26.418195 17727 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.516952 17546 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.016s	sys 0.017s
I20260709 13:03:26.485272 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:26.561425 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.550439 15126 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.637s (SENT)
I20260709 13:03:26.545890 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:26.570755 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.571211 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 510) took 2174 ms. Trace:
W20260709 13:03:26.574707 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.576292 15162 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:26.581200 15157 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:26.594213 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 354462us
W20260709 13:03:26.596676 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136197us
I20260709 13:03:26.583421 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:26.551832 17733 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:26.573575 17637 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.582088 17601 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:26.609884 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:26.618212 17724 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 13:03:26.618377 15082 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:26.618556 17717 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:03:26.619457 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:26.621140 17723 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:26.622866 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2371) took 2765 ms. Trace:
I20260709 13:03:26.623347 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 761) took 2806 ms. Trace:
I20260709 13:03:26.623420 15163 rpcz_store.cc:276] 0709 13:03:23.857936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.196239 (+1338303us) service_pool.cc:224] Handling call
0709 13:03:25.196655 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.452237 (+1255582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.622477 (+170240us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002882d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:26.622787 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2665064192}
I20260709 13:03:26.625232 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:26.625538 17709 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:03:26.625751 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2264) took 1573 ms. Trace:
W20260709 13:03:26.626101 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.631834 15236 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:26.616218 17712 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.199s
I20260709 13:03:26.598389 17553 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.221s
I20260709 13:03:26.632349 15084 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:26.612344 17637 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:26.635166 15233 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:26.653990 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.627024 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 509) took 2287 ms. Trace:
W20260709 13:03:26.651659 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57545.2us
I20260709 13:03:26.629267 17619 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:26.642524 17637 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:26.638806 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:26.683311 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 66144.9us
W20260709 13:03:26.645866 17718 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:03:26.660429 15159 rpcz_store.cc:276] 0709 13:03:24.339489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.727622 (+1388133us) service_pool.cc:224] Handling call
0709 13:03:25.728571 (+   949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.729326 (+   755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.729379 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.730500 (+  1121us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.334086 (+603586us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:26.552868 (+218782us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.573573 (+ 20705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.573654 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.626825 (+ 53171us) raft_consensus.cc:1685] finished
0709 13:03:26.626844 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.626970 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20377,"prepare.queue_time_us":293146,"raft.queue_time_us":251146,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":12170,"thread_start_us":808920,"threads_started":2,"wal-append.queue_time_us":796864}
I20260709 13:03:26.636770 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:26.639866 17714 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
W20260709 13:03:26.642096 17671 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.691s	user 0.018s	sys 0.000s
W20260709 13:03:26.654306 17592 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.238s	user 0.014s	sys 0.008s
I20260709 13:03:26.721165 16946 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:26.624148 15848 rpcz_store.cc:276] 0709 13:03:23.816776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.912502 (+ 95726us) service_pool.cc:224] Handling call
0709 13:03:23.912941 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.848594 (+935653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.848689 (+    95us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.849717 (+  1028us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.360707 (+1510990us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:26.368168 (+  7461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.368324 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.622971 (+254647us) raft_consensus.cc:1685] finished
0709 13:03:26.623001 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.623246 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209569,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":14924,"thread_start_us":2388779,"threads_started":2,"wal-append.queue_time_us":1568103}
I20260709 13:03:26.627429 15081 rpcz_store.cc:276] 0709 13:03:25.052107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.052270 (+   163us) service_pool.cc:224] Handling call
0709 13:03:25.070846 (+ 18576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.071184 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:26.415793 (+1344609us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.575931 (+160138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.589628 (+ 13697us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:26.625363 (+ 35735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.625480 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.625675 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1356276,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":1902,"raft.queue_time_us":19201,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":22140,"thread_start_us":1431366,"threads_started":2}
I20260709 13:03:26.639279 17553 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:26.738740 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:26.631994 17597 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:26.749573 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:26.773324 15841 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:26.752990 17592 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.754002 17668 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:26.757652 15847 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:26.636915 17710 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.771154 15160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1806) took 4098 ms (client timeout 4461 ms). Trace:
W20260709 13:03:26.784565 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.787563 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.682539 17619 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.787580 17719 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.522s	user 0.005s	sys 0.000s
W20260709 13:03:26.799995 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57010.9us
W20260709 13:03:26.800967 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.744736 17553 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:26.792021 15160 rpcz_store.cc:269] 0709 13:03:22.672105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.591131 (+919026us) service_pool.cc:224] Handling call
0709 13:03:26.770629 (+3179498us) spinlock_profiling.cc:243] Waited 2.86 s on lock 0x7b54000857d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:26.770847 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6573125376}
I20260709 13:03:26.571666 15154 rpcz_store.cc:276] 0709 13:03:24.396788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.751036 (+1354248us) service_pool.cc:224] Handling call
0709 13:03:25.751546 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.970389 (+218843us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:26.350007 (+379618us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.568482 (+218475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.570195 (+  1713us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:26.570414 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.570695 (+   281us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.571125 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2502,"prepare.queue_time_us":569060,"raft.queue_time_us":30690,"spinlock_wait_cycles":57600,"thread_start_us":504336,"threads_started":2}
I20260709 13:03:26.837486 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:26.838724 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 202176us
I20260709 13:03:26.839445 17546 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:26.840067 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:26.840809 17546 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:26.842432 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 806) took 2453 ms. Trace:
I20260709 13:03:26.842756 17546 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:26.838760 17704 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:26.844239 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.844449 17704 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:26.845068 15085 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:26.845532 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 676) took 1101 ms. Trace:
I20260709 13:03:26.845976 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.845927 15082 rpcz_store.cc:276] 0709 13:03:25.743720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.743877 (+   157us) service_pool.cc:224] Handling call
0709 13:03:25.744394 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.618938 (+874544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.845291 (+226353us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540014df54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:26.845477 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996107648}
I20260709 13:03:26.847222 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.847805 15082 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:26.848519 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:26.848801 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.849273 15839 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:26.851214 17277 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:26.852491 15084 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:26.809437 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:26.860234 15844 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:26.860364 15841 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:26.837358 17710 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:26.848752 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 499) took 3866 ms. Trace:
I20260709 13:03:26.864364 15157 rpcz_store.cc:276] 0709 13:03:22.982086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.192173 (+2210087us) service_pool.cc:224] Handling call
0709 13:03:25.192485 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.607033 (+1414548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.848525 (+241492us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540017dc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:26.848706 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3127300480}
I20260709 13:03:26.853168 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.866016 15087 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:26.852818 17749 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:26.853904 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.856626 17711 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:26.857075 17695 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:26.858019 17553 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:26.864820 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 291956us
I20260709 13:03:26.848623 17619 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:26.847857 15085 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:26.853808 17117 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:26.825152 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:26.826370 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.873083 15844 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:26.876169 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:26.842875 15840 rpcz_store.cc:276] 0709 13:03:24.388854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.390059 (+  1205us) service_pool.cc:224] Handling call
0709 13:03:24.390432 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.279807 (+889375us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:26.379328 (+1099521us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.592936 (+213608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.607013 (+ 14077us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:26.671017 (+ 64004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.842163 (+171146us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540053ded4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:26.842220 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.842374 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1066818,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":7820,"raft.queue_time_us":165,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":12200,"spinlock_wait_cycles":1969354624,"thread_start_us":1243339,"threads_started":2,"wal-append.queue_time_us":316306}
I20260709 13:03:26.816262 17592 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:26.825287 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 305126us
I20260709 13:03:26.881392 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:26.875106 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.878291 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.891973 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:26.876955 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:26.822932 17727 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:26.898101 15157 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:26.898576 15160 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:26.899489 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.890532 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:26.821796 17712 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:26.903587 15162 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:26.904793 15158 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:26.895772 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:26.923738 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.932485 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.934760 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:26.920933 15163 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:26.938625 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:26.904457 17486 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:03:26.940153 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:03:26.939079 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 99191.9us
I20260709 13:03:26.943997 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.934226 17704 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:26.944702 15154 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:26.945479 15162 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:26.945757 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.947965 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.921558 17484 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:26.945760 15233 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:26.964120 15157 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:26.961719 15236 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:26.947372 17553 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:26.970166 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.971179 15158 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:26.953167 17484 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.978225 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:26.977437 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:26.979553 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:26.981066 15234 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:26.981458 15085 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:26.986001 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.987774 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:26.992501 15079 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:26.997463 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:26.981333 17484 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:27.001109 15085 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:26.982568 17619 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.000660 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.004180 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.005905 15229 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.012856 17619 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.017244 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.026538 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:27.029737 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.004741 17725 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:27.028584 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.034600 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.037513 15234 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.040026 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.046293 15154 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:27.048803 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.049759 15235 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:27.051823 17671 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.054701 17733 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.048894 15232 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:27.056710 17710 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.057278 17710 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:27.054692 17758 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:03:27.058652 17484 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.061731 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.061801 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 495) took 4239 ms. Trace:
I20260709 13:03:27.063099 17671 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:27.062657 17484 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:27.062453 15161 rpcz_store.cc:276] 0709 13:03:22.821963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.952585 (+2130622us) service_pool.cc:224] Handling call
0709 13:03:25.006248 (+ 53663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.547988 (+1541740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.552299 (+  4311us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.635486 (+ 83187us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.820517 (+185031us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:26.820553 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.834032 (+ 13479us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.061523 (+227491us) raft_consensus.cc:1685] finished
0709 13:03:27.061540 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.061736 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":424659,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6831,"thread_start_us":1595522,"threads_started":2,"wal-append.queue_time_us":296722}
I20260709 13:03:27.066908 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.068441 15161 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:27.071847 17484 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.072583 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.075325 15088 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.076014 15237 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:27.076495 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:27.074687 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.077311 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.077745 15231 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:27.078104 15157 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:27.078975 15237 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.073352 15235 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:27.080663 17117 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:27.081255 17117 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:27.081058 17484 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.081729 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.082644 15232 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:27.073976 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:27.083848 15234 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.087059 17609 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:27.088617 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:27.089444 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49634.1us
I20260709 13:03:27.090700 16946 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:27.090732 17631 raft_consensus.cc:2764] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:03:27.091583 16946 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.092463 17631 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:27.093576 17631 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:27.094024 17631 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:27.095532 17631 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.096953 17631 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.097222 17603 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:27.087129 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 249781us
I20260709 13:03:27.077171 15230 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.038645 17695 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:27.034766 17619 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:27.040608 17711 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:27.103464 15234 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.104993 15163 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:27.107724 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:27.109120 15158 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:27.110682 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.111429 15848 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.112869 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:27.047652 17727 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:27.081775 17117 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:27.118263 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 244258us
I20260709 13:03:27.103873 17695 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.114748 15159 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.119974 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.115562 17484 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:27.121011 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:27.121611 17757 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:03:27.110600 17486 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:03:27.123240 17658 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:27.124586 15844 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:27.123890 15201 connection.cc:256] RPC call timeout handler was delayed by -1.0265505710000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:03:27.127662 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.141850 17658 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:27.132563 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.123960 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.134564 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:27.138492 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.134593 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.118065 17711 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.144994 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.141145 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.155277 15156 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:27.155504 15844 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:27.155949 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:27.125929 17695 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.161314 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.163077 15844 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:27.165387 17606 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:27.167516 17606 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:03:27.171326 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44458.7us
I20260709 13:03:27.159680 17597 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:27.173219 17597 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:27.174335 17117 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.174490 17597 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.175602 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.176466 15154 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:27.151816 15157 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:27.179351 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:27.179675 17760 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:03:27.141889 17703 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:27.200474 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.202785 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.130344 17619 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:27.199854 15201 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.694s (SENT)
W20260709 13:03:27.204826 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 85524.8us
I20260709 13:03:27.180035 15157 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.208335 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.211697 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.204905 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:27.215548 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 162155us
I20260709 13:03:27.146991 17486 consensus_queue.cc:1243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:27.214118 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.216300 15844 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:27.192988 17761 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:27.228950 15161 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:27.199476 17703 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.214476 17712 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:27.234968 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.245896 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:27.249416 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.221596 17671 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.218606 15841 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.214501 15840 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.255211 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:27.272114 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.268383 17117 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:27.271342 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 208673us
I20260709 13:03:27.279148 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.280474 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 518) took 1122 ms. Trace:
I20260709 13:03:27.275430 15847 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:27.284258 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.287806 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.284451 17671 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.292590 17704 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:27.293072 15157 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:27.286460 17117 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:27.281242 17484 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:27.284777 15845 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:27.289005 17712 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:27.290733 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:27.289932 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.297080 17431 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:27.290396 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.233174 17619 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.245266 17747 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:27.294461 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.296185 17712 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.299501 17546 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:27.299944 17546 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.300287 17619 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:27.300348 17546 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.295042 17117 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:27.243960 17761 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:27.281945 15160 rpcz_store.cc:276] 0709 13:03:26.157795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.948904 (+791109us) service_pool.cc:224] Handling call
0709 13:03:26.976189 (+ 27285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.976444 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.976481 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.979473 (+  2992us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:26.979522 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.979589 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.280225 (+300636us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b540037e950. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:27.280389 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25107200}
I20260709 13:03:27.304371 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:27.313933 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63234.2us
W20260709 13:03:27.321245 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.315670 16946 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.346529 15844 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.358896 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.297729 17431 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.347309 17117 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.375183 17783 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.375823 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 690) took 1318 ms. Trace:
I20260709 13:03:27.376474 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:27.377466 17553 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:27.377985 17553 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.378387 17553 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.380997 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.376308 15088 rpcz_store.cc:276] 0709 13:03:26.057029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.152755 (+ 95726us) service_pool.cc:224] Handling call
0709 13:03:27.349805 (+1197050us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54001b08d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:27.375691 (+ 25886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102256512}
W20260709 13:03:27.382357 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 209535us
W20260709 13:03:27.382956 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.381050 17619 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:27.386900 15157 raft_consensus.cc:2376] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3. Re-sending same reply.
W20260709 13:03:27.387784 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.357419 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 754) took 4100 ms (client timeout 1684 ms). Trace:
I20260709 13:03:27.375833 17783 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.391055 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.391283 16946 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:27.388900 15846 rpcz_store.cc:269] 0709 13:03:23.242846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.244793 (+  1947us) service_pool.cc:224] Handling call
0709 13:03:25.102103 (+1857310us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:27.279651 (+2177548us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54006d1d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:27.342805 (+ 63154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400280576}
I20260709 13:03:27.397739 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.400368 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.301236 17703 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.401145 15088 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:27.407006 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.392963 17619 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.406245 17783 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:27.407907 15163 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:27.401135 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.409216 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.410753 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.417346 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.419901 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.420616 15847 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:27.411983 17619 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.428244 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.430311 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.433772 15086 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:03:27.428550 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.434880 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.417392 17725 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.435719 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.436136 15080 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:27.387902 17695 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.438424 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.438489 15085 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:27.443063 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:27.440651 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:27.444741 15079 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:27.441460 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:27.440016 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59634.8us
W20260709 13:03:27.448098 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.449362 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.409893 17747 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:03:27.431290 17703 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.449976 15083 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 2.
I20260709 13:03:27.420042 17431 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:27.462908 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 158660us
I20260709 13:03:27.463567 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.464330 15160 raft_consensus.cc:2368] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5.
I20260709 13:03:27.465158 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.465379 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.465941 15161 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:27.466044 15160 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:27.466750 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.466954 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.467327 15161 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:27.468073 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.468159 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.465387 15847 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:27.469194 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.469794 15160 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:27.470309 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.470584 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.471194 15159 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:27.471297 15160 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:27.464309 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:27.473851 17781 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:27.473980 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.475064 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.475467 15154 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.475663 17768 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:27.475971 15847 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:27.475103 17704 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:27.476761 17704 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:27.477106 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.477877 15847 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:27.477731 17704 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.478777 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.480096 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.481022 15841 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:27.481200 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.482319 15845 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:27.482610 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.483752 15843 raft_consensus.cc:2410] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:27.485024 17598 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.390687 17711 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.492641 17622 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:03:27.493216 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.493825 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.494504 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.495217 15233 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:27.495072 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.496662 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:27.497120 17766 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:03:27.498296 17695 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:27.498473 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:27.504515 15234 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:27.468669 17553 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.507973 17553 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:27.494213 17622 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:27.511282 17710 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:27.510926 17117 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.511858 17710 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:27.461795 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.466073 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:27.534965 17117 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:27.516695 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 508) took 3182 ms. Trace:
I20260709 13:03:27.548803 15230 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:27.456041 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.541541 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59070.1us
W20260709 13:03:27.593361 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 119940us
I20260709 13:03:27.595065 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:27.595824 15842 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.512704 17668 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:27.600646 15080 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:27.450320 17725 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.623710 15163 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:27.512238 17710 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:27.747772 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:27.745173 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.545318 15155 rpcz_store.cc:276] 0709 13:03:24.334645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.261128 (+926483us) service_pool.cc:224] Handling call
0709 13:03:25.983046 (+721918us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:26.019625 (+ 36579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.085452 (+ 65827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.085497 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.141325 (+ 55828us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.519153 (+377828us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:26.704572 (+185419us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.742556 (+ 37984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.311316 (+568760us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b1800239ec0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419
0709 13:03:27.516279 (+204963us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b54003e1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:27.516329 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.516379 (+    50us) raft_consensus.cc:1685] finished
0709 13:03:27.516392 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.516530 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63066,"prepare.queue_time_us":132841,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":38434,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":62703,"spinlock_wait_cycles":297536384,"thread_start_us":462046,"threads_started":2,"wal-append.queue_time_us":406296}
I20260709 13:03:27.627087 15087 raft_consensus.cc:2410] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:27.515362 17601 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:27.625619 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:27.763479 15083 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:27.764325 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.712304 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:27.696439 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 230453us
I20260709 13:03:27.504580 15156 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:27.781232 15848 raft_consensus.cc:2368] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4.
W20260709 13:03:27.786151 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 94400.2us
I20260709 13:03:27.792496 15842 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:27.541465 17768 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.798554 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.810561 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.811668 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:27.818289 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.611409 17631 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:27.824199 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.612211 17668 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.828209 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.764192 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.828275 15230 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:27.829082 15082 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:27.831880 15161 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:27.833508 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.762954 17601 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:27.833581 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.715790 17703 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.851215 17703 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.836936 15236 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:27.452900 17761 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.838698 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:27.817574 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:27.846765 15086 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:27.848897 17484 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.850051 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.850678 15231 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:27.861805 15082 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:27.861967 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.866794 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.853693 17703 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:27.875443 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.877962 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.870430 17631 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:27.869194 15841 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.866302 17484 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.880520 17768 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.884605 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.897485 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.903096 15086 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:27.873109 17597 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:27.905339 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.860366 15083 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:27.864347 15840 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:27.909584 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.899916 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.876055 17725 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:27.912616 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.856518 17619 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.918538 15083 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:27.826098 17658 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.894106 17117 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:27.929297 15842 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:27.924701 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 124402us
I20260709 13:03:27.927646 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:27.923765 15230 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:27.931035 15086 raft_consensus.cc:2376] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5. Re-sending same reply.
I20260709 13:03:27.940994 17658 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.939532 17117 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.941329 17780 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 13:03:27.943085 17117 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:27.943188 17770 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.943949 17770 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.947345 17668 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:27.948197 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.948434 17668 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:27.951205 17631 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.928407 17597 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:27.953804 17597 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:27.959642 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:27.960281 15229 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.961203 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.961321 17791 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:27.962750 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:27.963425 15236 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 3.
I20260709 13:03:27.963927 15083 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:27.967547 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:27.968914 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 188149us
I20260709 13:03:27.968950 15230 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:27.971307 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:27.981701 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:27.982143 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.982543 15844 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7.
I20260709 13:03:27.982803 15154 raft_consensus.cc:2376] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6. Re-sending same reply.
I20260709 13:03:27.983376 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.983880 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:27.984040 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.984546 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.985385 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.987885 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:27.988592 15846 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:27.984752 15158 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7.
I20260709 13:03:27.993077 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:27.993391 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.984495 15155 raft_consensus.cc:2376] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6. Re-sending same reply.
I20260709 13:03:27.985183 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:27.962783 15841 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:28.009927 15083 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:27.914075 17710 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.012369 15086 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:28.008487 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:28.002842 17601 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:28.032145 17601 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:28.033569 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:28.034976 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:28.035372 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:28.035847 15157 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:28.028374 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:27.998097 17790 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:28.052129 17790 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:28.030157 17710 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:28.054831 17619 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:28.055581 17619 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:28.056465 17619 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:28.057365 17789 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:03:28.059310 17546 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
I20260709 13:03:28.059720 17546 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:27.997802 17798 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:03:28.042096 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.044932 17606 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:28.049063 15052 leader_election.cc:400] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7.
I20260709 13:03:28.073904 15052 leader_election.cc:403] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.049470 17117 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:28.062287 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:28.071547 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.080880 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43933us
I20260709 13:03:28.097399 15229 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:28.101871 16946 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.105048 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:28.073856 17606 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.113878 15228 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:28.114094 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.114423 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:28.121483 15236 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:28.115093 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:28.115527 17704 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:28.116593 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:28.083283 17778 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7.
I20260709 13:03:28.117617 15844 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:28.072536 15230 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:28.174785 16946 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:28.186110 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 328096us
I20260709 13:03:28.176057 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:28.188145 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:28.092895 17117 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:28.095469 17783 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.198559 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.204828 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83317.2us
I20260709 13:03:28.213236 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:28.209156 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.215309 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:28.123829 17606 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:28.120419 17770 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:28.210193 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:28.223855 15083 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:28.122903 17704 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:28.185633 17601 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:28.228351 16946 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:28.222865 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:28.234687 15080 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:28.229882 15203 leader_election.cc:336] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.609s (SENT)
I20260709 13:03:28.235534 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:28.231559 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 190444us
W20260709 13:03:28.231940 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 448312us
W20260709 13:03:28.232110 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44048.7us
I20260709 13:03:28.120586 17725 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
W20260709 13:03:28.239913 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 228211us
W20260709 13:03:28.240876 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.241079 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 127074us
I20260709 13:03:28.220607 17778 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:28.203809 17787 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:28.239202 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:28.247077 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 134436us
I20260709 13:03:28.231199 17546 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:28.235246 17704 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:28.249078 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:28.269348 17761 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:28.284152 17761 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.284556 17761 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:28.285027 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:28.286804 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:28.226910 17783 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.289011 17733 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:28.289467 17733 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:28.289876 17733 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:28.290606 17783 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:28.293219 17780 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:28.293655 17780 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:28.294034 17780 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.297060 17619 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:28.297425 15128 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.299592 17761 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.300123 17117 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.301218 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:28.301762 17761 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:28.303010 17778 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:28.242267 17601 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:28.306221 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63011.1us
I20260709 13:03:28.307628 15845 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:28.274649 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 239318us
W20260709 13:03:28.315457 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 253261us
I20260709 13:03:28.324910 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:28.329299 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:28.275362 17768 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.278651 17788 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:28.272891 17770 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.309226 17778 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:28.331668 15158 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:28.333137 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 114569us
I20260709 13:03:28.333058 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:28.335669 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:28.330238 15160 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:28.311419 15840 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:28.308569 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:28.354895 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.362059 15082 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:28.362658 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:28.364693 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.364748 15155 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:28.366726 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 62054.3us
I20260709 13:03:28.334805 17788 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.370520 15201 leader_election.cc:400] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4.
I20260709 13:03:28.363247 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:28.399230 17783 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:28.400935 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63357.7us
I20260709 13:03:28.343802 17778 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:28.402096 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.399132 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105034us
I20260709 13:03:28.421519 15080 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:28.333871 17785 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:03:28.369830 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.341890 17603 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:28.366564 17771 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:28.342060 17770 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:28.424389 15080 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:28.362104 17768 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:28.427488 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100611us
W20260709 13:03:28.455940 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 147643us
W20260709 13:03:28.425940 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.431766 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.457962 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.439522 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:28.455053 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:28.403159 15201 leader_election.cc:403] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.426105 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:28.401886 17788 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:28.391808 17486 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:28.393445 17814 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:28.473204 17768 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:28.489032 17809 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:03:28.490110 17780 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:28.492475 17780 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:28.493021 17816 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:28.492825 15846 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:28.439522 15161 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:03:28.482331 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56345.4us
W20260709 13:03:28.494086 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 123730us
I20260709 13:03:28.422626 17598 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:28.400249 17703 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.420809 17819 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:28.489236 17800 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:03:28.460003 17603 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:28.494850 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:28.342247 17761 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:03:28.399763 17783 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.479154 17770 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:28.481670 17801 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:03:28.501513 17710 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:28.504159 17603 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:28.550168 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 252832us
I20260709 13:03:28.497454 17619 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4.
I20260709 13:03:28.455396 17601 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:28.544059 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.481942 17771 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:28.622363 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 164927us
W20260709 13:03:28.553367 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58574.7us
I20260709 13:03:28.589712 15155 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:28.572548 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 701) took 1144 ms. Trace:
I20260709 13:03:28.571501 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2395) took 1150 ms. Trace:
I20260709 13:03:28.482200 17431 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:28.639202 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1823) took 1368 ms. Trace:
I20260709 13:03:28.562548 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:28.640755 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:28.363771 17597 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:28.650493 15162 rpcz_store.cc:276] 0709 13:03:27.271288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.745945 (+474657us) service_pool.cc:224] Handling call
0709 13:03:28.622080 (+876135us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:28.639119 (+ 17039us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264339840}
I20260709 13:03:28.557658 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 702) took 1127 ms. Trace:
I20260709 13:03:28.630472 15163 rpcz_store.cc:276] 0709 13:03:27.420621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.420956 (+   335us) service_pool.cc:224] Handling call
0709 13:03:28.571249 (+1150293us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:28.571407 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442406656}
W20260709 13:03:28.650594 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 211129us
I20260709 13:03:28.628364 15084 rpcz_store.cc:276] 0709 13:03:27.427859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.427981 (+   122us) service_pool.cc:224] Handling call
0709 13:03:28.572359 (+1144378us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54001b17d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:28.572493 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861513600}
I20260709 13:03:28.484504 17486 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.556308 17484 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
I20260709 13:03:28.496341 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:28.485458 17704 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.504736 17703 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:28.887816 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 236665us
I20260709 13:03:28.887941 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:28.889271 17747 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:28.889886 17747 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.890375 17747 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:28.892035 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.554177 17710 consensus_queue.cc:1243] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:28.892418 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:28.893698 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:28.894098 15155 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:28.895982 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:28.553978 17804 raft_consensus.cc:2764] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:03:28.835888 17761 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:28.565516 17797 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:03:28.922226 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 706) took 1297 ms. Trace:
I20260709 13:03:28.922722 15079 rpcz_store.cc:276] 0709 13:03:27.624541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.625827 (+  1286us) service_pool.cc:224] Handling call
0709 13:03:27.626177 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.744221 (+118044us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:27.744262 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.753008 (+  8746us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.989522 (+236514us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.989556 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.989618 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.921972 (+932354us) raft_consensus.cc:1685] finished
0709 13:03:28.921996 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.922154 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89251,"raft.queue_time_us":117887,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":313,"thread_start_us":236115,"threads_started":1,"wal-append.queue_time_us":461803}
I20260709 13:03:28.620945 17606 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.570547 17790 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.695056 15847 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
W20260709 13:03:28.555930 17807 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:03:28.724408 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 699) took 1297 ms. Trace:
I20260709 13:03:28.565008 15843 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:28.898970 17484 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.946637 15848 raft_consensus.cc:2368] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4.
I20260709 13:03:28.901763 17704 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:28.902199 17486 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:28.902046 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:28.547014 17783 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:28.686738 15203 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.546s (SENT)
I20260709 13:03:28.498037 17787 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.904333 17117 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:28.617951 17603 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:28.905073 15846 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:28.621325 17598 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:28.568482 17821 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:03:28.615525 17622 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:28.704528 15086 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:28.555258 17808 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:03:28.687971 17733 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:28.504755 17816 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:28.561372 17810 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
I20260709 13:03:28.911357 15081 rpcz_store.cc:276] 0709 13:03:27.429976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.430103 (+   127us) service_pool.cc:224] Handling call
0709 13:03:27.442985 (+ 12882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.544900 (+101915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.544947 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.658344 (+113397us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.124559 (+466215us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:28.124607 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.124669 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.557402 (+432733us) raft_consensus.cc:1685] finished
0709 13:03:28.557422 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.557583 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81537,"raft.queue_time_us":68386,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":61181,"thread_start_us":305865,"threads_started":1,"wal-append.queue_time_us":507120}
W20260709 13:03:28.913542 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 458585us
I20260709 13:03:28.927996 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 788) took 1449 ms. Trace:
I20260709 13:03:28.928673 15087 rpcz_store.cc:276] 0709 13:03:27.426795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.426922 (+   127us) service_pool.cc:224] Handling call
0709 13:03:28.708290 (+1281368us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400118854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:28.724322 (+ 16032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422061824}
W20260709 13:03:28.964041 15159 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.025s	sys 0.061s
W20260709 13:03:28.964699 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.962285 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:29.027254 15159 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.027s	sys 0.068s
W20260709 13:03:29.027508 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65314.7us
I20260709 13:03:28.943390 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 789) took 1404 ms. Trace:
I20260709 13:03:28.961681 15839 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:28.962841 17790 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:28.983790 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 297073us
W20260709 13:03:29.042459 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.043208 15203 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.670s (SENT)
I20260709 13:03:28.976572 17658 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:28.968135 17829 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.004796 17622 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:29.014753 15847 rpcz_store.cc:276] 0709 13:03:27.478337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.527837 (+ 49500us) service_pool.cc:224] Handling call
0709 13:03:28.562388 (+1034551us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:28.927927 (+365539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20514688}
I20260709 13:03:29.001681 17598 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:29.028299 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.039175 15844 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:29.047142 15201 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.569s (SENT)
I20260709 13:03:29.062831 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:29.015966 17788 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.003s
I20260709 13:03:29.063678 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.066882 15844 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:29.076469 15203 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.711s (SENT)
I20260709 13:03:29.036013 17431 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.054235 17711 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.065562 17695 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:03:29.082911 15087 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:29.083427 15081 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:28.985847 17787 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.084147 15847 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:29.113246 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49655.3us
I20260709 13:03:29.118186 15844 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:29.135551 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 88433.7us
W20260709 13:03:29.170446 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45402.1us
W20260709 13:03:29.091969 17813 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:03:29.090067 17791 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:29.180251 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.133690 17484 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.138051 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 828) took 1013 ms. Trace:
I20260709 13:03:29.187796 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:29.078691 17831 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:29.127827 17780 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:29.060377 17771 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 13:03:29.126848 17761 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.128441 17658 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:29.195870 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 114650us
I20260709 13:03:29.199095 15054 leader_election.cc:400] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7.
W20260709 13:03:29.205268 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 309348us
I20260709 13:03:29.179652 17770 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.075989 17788 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:29.054221 17622 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.030160 15843 rpcz_store.cc:276] 0709 13:03:27.538541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.539527 (+   986us) service_pool.cc:224] Handling call
0709 13:03:28.564373 (+1024846us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:28.564644 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.939833 (+375189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.941527 (+  1694us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21073792}
I20260709 13:03:29.192997 17831 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.194242 17606 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.192938 17668 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:29.194577 17768 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:29.176291 17671 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:03:29.224978 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 157840us
I20260709 13:03:29.225344 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 832) took 1109 ms. Trace:
I20260709 13:03:29.139314 17711 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:29.226198 15203 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.822s (SENT)
I20260709 13:03:29.210906 17780 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:29.249125 15079 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:29.249259 17780 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:29.252665 17622 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.191349 17431 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:03:29.257115 17761 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.259688 17733 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.252703 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:29.237943 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:29.238974 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 125515us
W20260709 13:03:29.239464 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51809.9us
W20260709 13:03:29.240710 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 104173us
I20260709 13:03:29.244725 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.242862 15845 rpcz_store.cc:276] 0709 13:03:28.115541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.226172 (+110631us) service_pool.cc:224] Handling call
0709 13:03:29.225116 (+998944us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b54002a12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:29.225285 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70806912}
I20260709 13:03:29.242394 17768 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.246891 15084 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:29.247567 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.248062 17117 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:29.266215 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:29.266849 15084 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
W20260709 13:03:29.270197 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44039us
W20260709 13:03:29.271268 15203 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.887s (SENT)
I20260709 13:03:29.271766 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:29.278604 15203 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.900s (SENT)
I20260709 13:03:29.287027 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.276901 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:29.288280 15081 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.237979 17831 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.290390 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:29.292418 15083 raft_consensus.cc:2376] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:29.232762 17770 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:29.293484 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:29.302511 15087 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:29.303437 17711 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:29.296392 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67026.6us
I20260709 13:03:29.305896 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:29.307082 17778 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.871s	user 0.009s	sys 0.000s
I20260709 13:03:29.307547 17484 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:29.307524 17778 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.274575 17771 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:29.265681 17768 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.291349 17838 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:29.307005 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.311602 17839 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:29.312068 17839 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.312425 17839 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:29.312646 17832 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:29.190258 15841 rpcz_store.cc:276] 0709 13:03:28.114203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.122889 (+  8686us) service_pool.cc:224] Handling call
0709 13:03:28.227593 (+104704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.275731 (+ 48138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.281195 (+  5464us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.392000 (+110805us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.897042 (+505042us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:28.897095 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.897178 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.104596 (+207418us) raft_consensus.cc:1685] finished
0709 13:03:29.104624 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.127634 (+ 23010us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27854,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":31074,"thread_start_us":345328,"threads_started":1,"wal-append.queue_time_us":558030}
I20260709 13:03:29.275321 17840 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:29.277405 17603 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.315615 17788 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:29.316174 17787 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:29.321298 17798 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:29.320932 17658 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.322010 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1680) took 1386 ms. Trace:
I20260709 13:03:29.322391 15082 rpcz_store.cc:276] 0709 13:03:27.935955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.936354 (+   399us) service_pool.cc:224] Handling call
0709 13:03:29.321835 (+1385481us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54000dbbd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:29.321952 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895377152}
W20260709 13:03:29.301501 17836 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:03:29.324415 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.335227 15081 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:29.335813 15201 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.711s (SENT)
I20260709 13:03:29.265892 17671 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.268798 17606 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.266175 17117 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:29.339691 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.308248 17484 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:29.350837 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103332us
I20260709 13:03:29.210741 17816 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.353083 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.354154 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:29.361351 15082 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.324191 17668 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:03:29.371495 15087 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:29.337728 17804 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:29.375056 15201 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.717s (SENT)
W20260709 13:03:29.381587 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 90787.8us
I20260709 13:03:29.374130 17788 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:29.356247 17484 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.391913 16946 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:29.393332 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.397079 17829 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:29.388332 17598 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:29.400552 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 121993us
I20260709 13:03:29.391664 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:29.409855 15203 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.012s (SENT)
I20260709 13:03:29.409579 17117 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.402190 15079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:29.410367 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.411029 15079 raft_consensus.cc:606] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:29.411469 15079 raft_consensus.cc:614] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:29.402570 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:29.408807 17829 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.409400 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:29.412241 15081 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:29.351691 17671 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.410355 17598 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:29.413843 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 797) took 1484 ms (client timeout 1717 ms). Trace:
W20260709 13:03:29.426281 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51282.2us
W20260709 13:03:29.426626 15842 rpcz_store.cc:269] 0709 13:03:27.929496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.391394 (+1461898us) service_pool.cc:224] Handling call
0709 13:03:29.413774 (+ 22380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.412655 17703 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.351976 17838 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:03:29.431494 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.08703e+06us
I20260709 13:03:29.431711 17770 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.433106 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:29.413560 17851 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 13:03:29.433502 17747 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 13:03:29.444240 17747 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.445343 17829 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.450050 17117 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:29.451378 17818 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.154s	user 0.000s	sys 0.005s
W20260709 13:03:29.456502 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63231us
W20260709 13:03:29.425374 15203 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.031s (SENT)
I20260709 13:03:29.458257 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.418123 15082 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:29.436209 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 531) took 1105 ms. Trace:
W20260709 13:03:29.434006 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 793306us
I20260709 13:03:29.437635 15155 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:29.438104 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.439157 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.460939 15160 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:29.438815 15158 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:29.440191 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:29.441856 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.465787 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.466121 17783 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.467746 17704 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:29.468142 17704 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.468353 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:29.468667 17778 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.440091 17851 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:29.459213 15159 rpcz_store.cc:276] 0709 13:03:28.330606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.332291 (+  1685us) service_pool.cc:224] Handling call
0709 13:03:28.342653 (+ 10362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.342870 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:29.027742 (+684872us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.435884 (+408142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.435935 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.435988 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.436052 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.436158 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":252429,"prepare.queue_time_us":95359,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":254842,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":789763,"threads_started":2,"wal-append.queue_time_us":234472}
I20260709 13:03:29.413338 17606 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.442158 17486 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.431574 17725 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.470160 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:29.437772 17814 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.441218 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.439451 17823 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.441290 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:29.439561 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:29.443379 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:29.512118 15162 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:29.437996 17831 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:29.472369 17840 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.435631 17597 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.434878 17819 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:29.424592 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:29.427335 15201 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.123s (SENT)
I20260709 13:03:29.478226 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 773) took 1185 ms. Trace:
I20260709 13:03:29.465049 17710 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:29.480799 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:29.529457 15158 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:29.529538 15842 pending_rounds.cc:85] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 7
W20260709 13:03:29.485381 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45608.2us
W20260709 13:03:29.485831 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60493.2us
I20260709 13:03:29.484894 17851 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:29.488620 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:29.494872 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.496196 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:29.534235 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45739.2us
I20260709 13:03:29.534395 15160 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:29.468297 17783 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.498260 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:29.511307 15154 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:29.510339 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51970.4us
I20260709 13:03:29.415683 17804 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:03:29.509174 15163 pending_rounds.cc:85] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 4
I20260709 13:03:29.526443 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:29.526623 15846 rpcz_store.cc:276] 0709 13:03:28.293136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.553969 (+260833us) service_pool.cc:224] Handling call
0709 13:03:28.554285 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.004812 (+450527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.477824 (+473012us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540041c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:29.478134 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806249856}
I20260709 13:03:29.481756 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:29.540426 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.427075 17671 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.533128 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.535380 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:29.541523 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.535684 15842 pending_rounds.cc:107] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:03:29.540076 15163 pending_rounds.cc:107] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:03:29.539217 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.541467 15159 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:29.543290 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:29.560542 15163 raft_consensus.cc:2889] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.543159 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.561393 16946 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:29.561728 15842 raft_consensus.cc:2889] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.541970 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:29.564057 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 139607us
W20260709 13:03:29.575016 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 93354.4us
I20260709 13:03:29.565281 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.565982 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.565864 17824 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:29.568497 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.576857 15234 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.577205 15231 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.568420 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.579948 15230 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:29.510685 17761 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:29.531499 17819 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:29.576646 15236 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:29.589020 15236 pending_rounds.cc:85] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
I20260709 13:03:29.589418 15236 pending_rounds.cc:107] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:03:29.589666 15236 raft_consensus.cc:2889] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.583734 15233 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:29.591887 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.535113 17823 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:29.537221 17781 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.587636 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.559514 17787 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:29.595726 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.602312 15162 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:03:29.597851 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 158427us
I20260709 13:03:29.599118 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.570914 17838 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:29.576711 17791 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:29.576422 17725 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:29.642849 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.643535 15236 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:29.635152 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:29.636473 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 93314.7us
I20260709 13:03:29.559700 17658 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:29.583608 17770 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:29.529019 17597 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.634456 15232 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:29.527442 17839 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.610355 17783 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:29.653801 15234 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:29.595302 17819 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:29.693456 15203 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.590s (SENT)
I20260709 13:03:29.513274 17814 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.636755 16946 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.600399 17787 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:29.694365 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:29.694587 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:29.609817 17788 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:29.610256 17829 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:29.633924 17703 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:29.698994 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 271675us
W20260709 13:03:29.763332 15201 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.803s (SENT)
I20260709 13:03:29.763952 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:29.710482 15235 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:29.710434 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 111363us
W20260709 13:03:29.732239 15203 leader_election.cc:336] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.785s (SENT)
I20260709 13:03:29.765247 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.707656 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.709488 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:29.708165 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:29.697150 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 86484.5us
I20260709 13:03:29.766520 15232 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:29.738142 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:29.767987 15201 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.861s (SENT)
I20260709 13:03:29.768260 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.768761 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:29.698711 17770 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.707212 17815 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.230s
I20260709 13:03:29.707962 17851 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:29.746266 17831 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.746842 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:29.757222 17790 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.699s	user 0.007s	sys 0.003s
W20260709 13:03:29.750618 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 175020us
I20260709 13:03:29.776011 17606 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.610522 17781 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.758999 15159 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:29.600157 17484 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.770044 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.774750 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:29.775257 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:29.776094 15157 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:29.697971 17783 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:29.778241 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 207946us
W20260709 13:03:29.780619 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48422.3us
I20260709 13:03:29.748447 17787 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.757222 17671 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:29.812378 15846 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:03:29.651811 17838 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.748788 17830 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.301s
I20260709 13:03:29.792066 15234 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:29.797712 15232 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:29.782173 16946 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:29.799968 15230 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
W20260709 13:03:29.801541 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.787221 17829 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:29.777809 17780 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:29.774832 17844 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:29.809733 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.804049 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:29.778116 17826 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:29.708580 17603 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:29.802340 15229 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:29.783866 17622 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:29.834975 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.836498 15234 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:29.838452 17804 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:29.780005 17824 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:03:29.843921 17872 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:29.802802 17851 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:29.805771 17815 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:03:29.809780 17783 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:30.030507 15235 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:29.994797 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 224804us
W20260709 13:03:30.029454 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.035009 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:29.973776 15203 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.713s (SENT)
I20260709 13:03:30.035326 15160 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:29.804939 17788 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:29.814525 17798 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:30.033993 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:29.817644 17671 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.044152 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:29.815013 17703 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:03:29.993505 17852 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:03:29.813055 17486 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:30.077025 17770 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:29.942368 17631 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.146s
W20260709 13:03:30.080998 17861 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:03:30.029067 17844 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.994117 17484 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:29.891166 17725 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:30.082238 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:29.803172 17864 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.294s
I20260709 13:03:30.494467 15230 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:03:30.501928 15230 pending_rounds.cc:85] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
W20260709 13:03:30.130501 15231 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.000s
I20260709 13:03:30.502333 15230 pending_rounds.cc:107] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 4.6
W20260709 13:03:30.464141 15229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1822) took 2355 ms (client timeout 2062 ms). Trace:
I20260709 13:03:30.503121 15230 raft_consensus.cc:2889] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:30.454734 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:30.449389 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.506414 15231 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:30.184046 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149094us
W20260709 13:03:30.235635 17117 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.785s	user 0.012s	sys 0.000s
I20260709 13:03:30.030877 17780 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.503243 15229 rpcz_store.cc:269] 0709 13:03:28.108954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.114869 (+  5915us) service_pool.cc:224] Handling call
0709 13:03:30.463866 (+2348997us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400295c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:30.464035 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3862061312}
W20260709 13:03:30.233744 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 576746us
W20260709 13:03:30.098013 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.006s	sys 0.022s
W20260709 13:03:30.044420 17768 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
W20260709 13:03:30.528952 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 494980us
I20260709 13:03:30.531174 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:29.913748 17867 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:30.522194 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.043s	sys 0.080s
I20260709 13:03:30.517987 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:30.537384 15844 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.538619 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 718) took 1244 ms. Trace:
W20260709 13:03:30.300289 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 793) took 2478 ms (client timeout 1569 ms). Trace:
I20260709 13:03:30.232985 17871 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:30.236609 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 776) took 1807 ms. Trace:
W20260709 13:03:30.515244 17819 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.000s
W20260709 13:03:30.458775 17873 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.525s	user 0.003s	sys 0.000s
I20260709 13:03:29.989372 17839 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:30.036799 17826 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:30.536885 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 996454us
I20260709 13:03:30.539017 15083 rpcz_store.cc:276] 0709 13:03:29.294328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.294511 (+   183us) service_pool.cc:224] Handling call
0709 13:03:29.350838 (+ 56327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.351174 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:29.841243 (+490069us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.236886 (+395643us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.463474 (+226588us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:30.538260 (+ 74786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.538309 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:30.538354 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.538416 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.538561 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":738681,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":762357,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":236,"spinlock_wait_cycles":58227200,"thread_start_us":1160183,"threads_started":4,"wal-append.queue_time_us":425507}
W20260709 13:03:30.597536 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59970.4us
I20260709 13:03:30.612473 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:30.464172 17877 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:03:30.618242 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:30.527613 15846 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:30.044178 17592 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.597368 15848 rpcz_store.cc:269] 0709 13:03:27.821503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.829562 (+  8059us) service_pool.cc:224] Handling call
0709 13:03:30.300014 (+2470452us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:30.300201 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2488450816}
I20260709 13:03:30.182821 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:30.599108 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 57435.8us
I20260709 13:03:30.619807 15234 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:30.620142 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 646370us
I20260709 13:03:30.297066 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 838) took 1001 ms. Trace:
I20260709 13:03:30.601220 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:30.621481 15231 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:30.416546 17814 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:30.169322 17781 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:30.630137 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:30.637976 15846 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:30.601006 17484 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.095274 17858 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 13:03:30.602705 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:30.641247 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:30.642138 15846 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:30.603016 15232 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.001s
I20260709 13:03:30.529404 17830 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
W20260709 13:03:30.516291 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.604457 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:30.509039 17844 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:30.523940 17780 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:30.534322 17671 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.608228 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:30.515108 17878 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:30.609891 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 778) took 1172 ms. Trace:
I20260709 13:03:30.093377 17597 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:30.515578 17870 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.230824 17815 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.371s
I20260709 13:03:30.637647 17592 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:30.620771 15203 leader_election.cc:336] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.825s (SENT)
I20260709 13:03:30.598901 17872 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:30.624959 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:30.487717 17783 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.536752 17867 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:30.630069 17798 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.630384 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:30.635720 17787 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.815s	user 0.007s	sys 0.003s
I20260709 13:03:30.636282 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:30.604086 17703 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:30.599347 17871 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:30.639586 15844 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:30.084539 17824 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:30.642307 15234 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
W20260709 13:03:30.642452 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.016s	sys 0.034s
W20260709 13:03:30.742904 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.016s	sys 0.034s
W20260709 13:03:30.743234 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 976247us
I20260709 13:03:30.743970 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:30.083623 17788 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:30.644140 15229 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:30.642794 15232 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:30.646126 15082 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:30.644765 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1832) took 1066 ms. Trace:
W20260709 13:03:30.649349 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46743.7us
I20260709 13:03:30.541220 17768 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:30.663297 15842 rpcz_store.cc:276] 0709 13:03:29.437486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.511055 (+ 73569us) service_pool.cc:224] Handling call
0709 13:03:29.529343 (+ 18288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.573204 (+ 43861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.573235 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.573696 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.644487 (+ 70791us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.644513 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.644564 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.609568 (+965004us) raft_consensus.cc:1685] finished
0709 13:03:30.609597 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.609805 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":70451,"threads_started":1,"wal-append.queue_time_us":205139}
I20260709 13:03:30.090313 17829 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.670692 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:30.303671 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:30.045629 17838 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.681407 15848 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:30.673794 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:30.642520 17814 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.676251 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:30.848474 15846 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:30.613862 17819 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.663241 17878 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.667658 17859 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:30.738154 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1683) took 2672 ms. Trace:
I20260709 13:03:30.674268 17770 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:30.738675 15847 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 7. Ops: []
I20260709 13:03:30.647143 17484 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:30.739984 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.679557 17867 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:30.725214 17781 raft_consensus.cc:1064] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:03:30.865523 15086 rpcz_store.cc:276] 0709 13:03:28.013329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.088123 (+ 74794us) service_pool.cc:224] Handling call
0709 13:03:28.105612 (+ 17489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.005099 (+899487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.681101 (+1676002us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540025ba54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:30.681309 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376653696}
W20260709 13:03:30.842669 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 98413us
I20260709 13:03:30.774329 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 551) took 1067 ms. Trace:
I20260709 13:03:30.774712 15844 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:03:30.777926 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 552) took 1069 ms. Trace:
I20260709 13:03:30.677199 17815 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.863660 15231 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:03:30.942823 15842 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:30.775058 15228 rpcz_store.cc:276] 0709 13:03:29.578209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.611608 (+ 33399us) service_pool.cc:224] Handling call
0709 13:03:29.692409 (+ 80801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.733140 (+ 40731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.733175 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.751742 (+ 18567us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.805515 (+ 53773us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:29.805549 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.805619 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.457889 (+652270us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:30.638278 (+180389us) raft_consensus.cc:1685] finished
0709 13:03:30.638297 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.644701 (+  6404us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50809,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":14150,"spinlock_wait_cycles":17347328,"thread_start_us":17001,"threads_started":1,"wal-append.queue_time_us":67490}
I20260709 13:03:30.942890 15234 rpcz_store.cc:276] 0709 13:03:29.708158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.777126 (+1068968us) service_pool.cc:224] Handling call
0709 13:03:30.777426 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.777605 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.777632 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.777668 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:30.777707 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.777763 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.777865 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.940891 15232 rpcz_store.cc:276] 0709 13:03:29.707087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.775010 (+ 67923us) service_pool.cc:224] Handling call
0709 13:03:30.774270 (+999260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.742718 17824 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:30.955737 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1839) took 1050 ms. Trace:
I20260709 13:03:30.955660 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:30.777263 17790 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:30.842047 17864 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:30.627737 15843 rpcz_store.cc:276] 0709 13:03:29.295643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.324365 (+ 28722us) service_pool.cc:224] Handling call
0709 13:03:29.362579 (+ 38214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.362990 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.363022 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.363522 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.571041 (+207519us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:29.583401 (+ 12360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.597565 (+ 14164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.233307 (+635742us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b540036de20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f959dbe 00007f0d3f964930 00007f0d3f9648d8 00007f0d3f964858 00007f0d3f964813 00007f0d3f963a9c 00007f0d4584f626 00007f0d45849dd8 00007f0d3f9a07cb 00007f0d3f9a32c0
0709 13:03:30.233741 (+   434us) raft_consensus.cc:1685] finished
0709 13:03:30.233767 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.296964 (+ 63197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1415,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":10821,"spinlock_wait_cycles":12691200,"thread_start_us":167496,"threads_started":1,"wal-append.queue_time_us":209127}
I20260709 13:03:30.964679 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:30.772604 17597 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:30.957291 15231 rpcz_store.cc:276] 0709 13:03:29.893168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.858654 (+965486us) service_pool.cc:224] Handling call
0709 13:03:30.859076 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.943105 (+ 84029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.943292 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.964468 15229 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:30.965473 15232 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:30.966807 15161 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:30.858611 17592 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:30.989897 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45186.3us
W20260709 13:03:30.995577 15051 connection.cc:256] RPC call timeout handler was delayed by -1.0308376169999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:03:30.749848 17881 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:30.725672 17798 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.991405 15848 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:31.002099 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 400473us
I20260709 13:03:30.728765 17826 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.958671 17892 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.939680 17878 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.990684 17864 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.990224 17830 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:30.680958 17870 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:30.965111 17770 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:30.865504 17814 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.018560 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 397785us
I20260709 13:03:31.019955 15228 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:30.997486 17597 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:30.682747 17725 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.007823 17859 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:31.033401 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:31.017925 17703 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:31.030928 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.020157 15841 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:30.953614 17787 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.050160 15203 connection.cc:256] RPC call timeout handler was delayed by -1.0047696480000923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 13:03:31.020807 17881 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:31.059978 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.061180 15203 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.944s (SENT)
I20260709 13:03:31.064025 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:31.025195 17864 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:31.065059 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.065786 17879 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:30.956588 17815 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.070184 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:31.074239 17819 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:31.075886 17888 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:03:31.078523 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57158.7us
I20260709 13:03:30.602090 15840 rpcz_store.cc:276] 0709 13:03:28.427163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.909035 (+481872us) service_pool.cc:224] Handling call
0709 13:03:29.227701 (+318666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.255818 (+ 28117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.255859 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.256465 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.389782 (+133317us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.390162 (+   380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.411201 (+ 21039us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.424217 (+ 13016us) raft_consensus.cc:1685] finished
0709 13:03:29.426562 (+  2345us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.096384 (+669822us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54002a12d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:30.234791 (+138407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10372,"raft.queue_time_us":11736,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":727804288,"thread_start_us":119159,"threads_started":1,"wal-append.queue_time_us":161683,"wal-append.run_cpu_time_us":1640,"wal-append.run_wall_time_us":1001671}
I20260709 13:03:30.955399 17871 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.069218 17484 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.060943 17894 raft_consensus.cc:2764] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:03:31.101658 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:31.112023 15203 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.012s (SENT)
I20260709 13:03:31.112692 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:31.119109 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1831) took 1549 ms. Trace:
I20260709 13:03:31.119602 15233 rpcz_store.cc:276] 0709 13:03:29.570179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.611715 (+ 41536us) service_pool.cc:224] Handling call
0709 13:03:29.612066 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.612412 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.612440 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.641717 (+ 29277us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.466986 (+825269us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b1800437920. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:30.646373 (+179387us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:30.646413 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.646493 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.114556 (+468063us) raft_consensus.cc:1685] finished
0709 13:03:31.114575 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.119033 (+  4458us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24843,"raft.queue_time_us":36762,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":42485,"spinlock_wait_cycles":5147392,"thread_start_us":179174,"threads_started":1,"wal-append.queue_time_us":1030817}
I20260709 13:03:31.123684 17798 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.124873 15158 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:31.125068 17902 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.125324 15158 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:31.125679 17902 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.126184 15158 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.127285 15158 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:31.030690 17824 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.128520 17824 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:31.130474 17815 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.131398 17901 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.132018 17901 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.133905 17898 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.070377 17881 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.134644 17901 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.136235 17881 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.124712 17903 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:31.138969 17780 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.139541 17859 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.140545 17882 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.141474 17814 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.137943 17894 raft_consensus.cc:2764] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:03:31.050001 17892 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.136719 17768 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.132640 17770 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.133971 17870 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.139048 17903 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.032189 17830 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.130395 17897 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.118395 17783 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.194900 17798 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.203611 17897 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.204484 17619 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:31.273319 17619 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:31.274290 17619 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.180661 17815 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:31.195147 17882 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:31.208218 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.272617 17787 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:31.117012 17703 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.099532 17597 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.134694 17898 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.475636 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.191264 17879 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:03:31.505976 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.010s	sys 0.011s
W20260709 13:03:31.658463 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.010s
W20260709 13:03:31.664793 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.013s	sys 0.014s
I20260709 13:03:31.371332 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 801) took 1330 ms. Trace:
W20260709 13:03:31.665280 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.294121 15158 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:31.665512 15847 rpcz_store.cc:276] 0709 13:03:30.040882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.041047 (+   165us) service_pool.cc:224] Handling call
0709 13:03:30.232234 (+191187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.939919 (+707685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.371044 (+431125us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540083e5d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:31.371269 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164237568}
W20260709 13:03:31.664899 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.003s	sys 0.010s
W20260709 13:03:31.666818 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 566494us
I20260709 13:03:31.667025 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1844) took 2223 ms. Trace:
I20260709 13:03:31.668689 15158 rpcz_store.cc:276] 0709 13:03:29.443463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.532446 (+ 88983us) service_pool.cc:224] Handling call
0709 13:03:29.532764 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.124740 (+1591976us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540015b420. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:03:31.666282 (+541542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.666972 (+   690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531344768}
W20260709 13:03:31.674685 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 573097us
W20260709 13:03:31.702934 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.703536 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.664364 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.013s
W20260709 13:03:31.726728 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.000s	sys 0.013s
W20260709 13:03:31.727139 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 656990us
I20260709 13:03:31.728565 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:31.730458 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:31.591205 17867 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.626s	user 0.003s	sys 0.007s
I20260709 13:03:31.203045 17830 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:31.631172 17878 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.601s	user 0.003s	sys 0.005s
I20260709 13:03:31.245350 17768 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.203639 17905 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.663645 17838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.014s
W20260709 13:03:31.472828 17844 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.000s
W20260709 13:03:31.654989 17884 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.007s	sys 0.014s
I20260709 13:03:31.370085 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:31.365682 17907 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:31.369781 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:31.642198 17892 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:31.626802 17896 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.771308 17597 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:31.773573 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:31.776212 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.017s	sys 0.027s
W20260709 13:03:31.848141 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.017s	sys 0.027s
W20260709 13:03:31.742471 17863 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:03:31.745877 17791 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:31.852118 17791 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.852521 17791 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:31.853310 17864 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.000s
I20260709 13:03:31.853832 17864 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.856185 17864 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.197575 17900 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.607s
I20260709 13:03:31.278122 17798 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.781469 17838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.000s	sys 0.014s
I20260709 13:03:31.746320 17908 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:31.771476 17867 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.866442 17913 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.867195 17913 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.867826 17867 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.869807 17898 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.870510 17898 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.871210 17898 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.738476 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.850274 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1839) took 2408 ms. Trace:
W20260709 13:03:31.850943 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.015s	sys 0.026s
I20260709 13:03:31.772344 17878 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.838747 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.875010 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.016s	sys 0.026s
I20260709 13:03:31.874820 15157 rpcz_store.cc:276] 0709 13:03:29.442142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.466700 (+ 24558us) service_pool.cc:224] Handling call
0709 13:03:29.467007 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.797655 (+330648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.850002 (+2052347us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540017f054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:31.850207 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710014208}
W20260709 13:03:31.875691 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 763583us
I20260709 13:03:31.857275 17903 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.880455 17619 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.882007 17878 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.850801 17747 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.781412 17768 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.861513 17671 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.185s	user 0.010s	sys 0.003s
W20260709 13:03:31.884243 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 117208us
I20260709 13:03:31.860898 17864 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:31.899724 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 559) took 1364 ms. Trace:
W20260709 13:03:31.896740 15203 leader_election.cc:336] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.611s (SENT)
I20260709 13:03:31.902288 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:31.894292 17914 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.885644 17892 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.902834 17916 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:31.906312 17874 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:31.906286 17916 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.907063 17874 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:31.908344 17874 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.902878 17619 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.904914 17914 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.910310 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.910938 17546 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:31.904187 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:31.904006 15229 rpcz_store.cc:276] 0709 13:03:30.535740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.004925 (+469185us) service_pool.cc:224] Handling call
0709 13:03:31.005229 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.898073 (+892844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.898107 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.898156 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:31.898199 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.899423 (+  1224us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54002d4954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:31.899489 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.899649 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052853632}
I20260709 13:03:31.790807 17905 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.796957 17844 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:31.920598 15203 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 3.251s (SENT)
I20260709 13:03:31.924986 17913 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.925426 17710 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:31.926007 17710 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:31.927371 17710 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.928138 17768 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.925932 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:31.911530 17546 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:31.928766 17710 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:31.916100 17671 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.930397 17844 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.953110 17671 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:31.939198 17826 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:31.953817 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:31.954747 17864 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:31.934901 17898 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.921653 17747 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:31.905360 17892 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.858300 17896 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:31.958792 15086 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:31.931926 17905 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:31.905679 17838 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:31.919616 17622 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:31.964964 17546 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:31.940320 17824 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.000s
I20260709 13:03:31.947010 17908 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.972144 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:31.948501 17900 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:32.005357 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:32.005745 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:32.314473 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:32.226866 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.015483 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:32.422514 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.038478 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:32.423574 15085 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:03:32.255369 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.425668 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.037479 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:32.070264 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 116533us
I20260709 13:03:32.083737 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:32.313730 15233 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:32.099660 15080 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:32.249918 15810 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.970s (SENT) [suppressed 52 similar messages]
W20260709 13:03:32.111447 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.432261 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.036150 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1838) took 2394 ms. Trace:
I20260709 13:03:32.070276 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.436458 15087 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:32.295686 15088 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:32.417093 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:32.440747 15813 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.555s (SENT)
I20260709 13:03:32.441349 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:32.422791 15081 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:32.434448 15231 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:31.969288 17907 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:32.467725 15054 connection.cc:256] RPC call timeout handler was delayed by -1.1938423200000443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:03:32.481009 15813 leader_election.cc:336] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.680s (SENT)
W20260709 13:03:32.526418 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45464.6us
W20260709 13:03:32.527204 15813 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.775s (SENT)
I20260709 13:03:32.527645 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:32.433655 15230 rpcz_store.cc:276] 0709 13:03:29.641676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.840625 (+198949us) service_pool.cc:224] Handling call
0709 13:03:30.467287 (+626662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.522502 (+ 55215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.035883 (+1513381us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400295c50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:32.036053 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440406912}
I20260709 13:03:32.503247 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:32.555866 15080 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:31.989765 17859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.009s	sys 0.014s
I20260709 13:03:31.982642 17867 raft_consensus.cc:2764] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:03:32.421315 15084 raft_consensus.cc:2376] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7. Re-sending same reply.
W20260709 13:03:32.563172 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.996001 17829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.004s	sys 0.023s
I20260709 13:03:32.561087 15083 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.437955 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.572551 15087 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:32.567898 15813 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.866s (SENT)
I20260709 13:03:32.574724 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:32.220455 17823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.476s	user 0.015s	sys 0.030s
I20260709 13:03:32.576390 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.312312 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:32.150964 17922 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:03:32.579473 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:32.041055 17925 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:32.582207 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:32.562762 17859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.420s	user 0.012s	sys 0.031s
I20260709 13:03:32.557536 17874 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.346383 17896 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:32.597018 14982 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:32.305155 17791 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.628083 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:32.572028 17829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.366s	user 0.013s	sys 0.039s
W20260709 13:03:32.578500 17823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.868s	user 0.019s	sys 0.034s
I20260709 13:03:32.385948 17864 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.608134 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.463024 17892 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:32.613842 17790 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:32.614708 17929 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.576121 17844 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.577359 17546 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.038194 17838 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.566273 17907 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.667286 17838 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:32.664078 17844 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.670668 17844 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.670863 15079 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:32.654716 15080 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:32.656003 15081 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:32.672379 17780 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:32.673558 15081 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:32.659976 15228 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:32.537743 17908 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:32.617517 17926 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:32.498139 17900 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:32.646973 17823 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:32.617796 17925 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:32.679698 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 429798us
W20260709 13:03:32.678490 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.675184 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 95811.3us
I20260709 13:03:32.689853 15840 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:03:32.690562 15840 raft_consensus.cc:606] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:32.693259 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 803) took 2651 ms. Trace:
I20260709 13:03:32.693831 15841 rpcz_store.cc:276] 0709 13:03:30.041668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.046565 (+  4897us) service_pool.cc:224] Handling call
0709 13:03:30.602156 (+555591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.063389 (+461233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.692877 (+1629488us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54005a55d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:32.693175 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922181376}
I20260709 13:03:32.697700 15839 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:32.700109 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 848) took 1680 ms. Trace:
I20260709 13:03:32.701459 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1848) took 3257 ms. Trace:
I20260709 13:03:32.701954 15159 rpcz_store.cc:276] 0709 13:03:29.444514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.570506 (+125992us) service_pool.cc:224] Handling call
0709 13:03:29.595225 (+ 24719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.788540 (+193315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.701181 (+2912641us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540037f854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:32.701386 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357354240}
I20260709 13:03:32.705655 15847 raft_consensus.cc:2410] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:32.703929 17826 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.750s	user 0.004s	sys 0.007s
I20260709 13:03:32.700572 15839 rpcz_store.cc:276] 0709 13:03:31.019970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.024462 (+  4492us) service_pool.cc:224] Handling call
0709 13:03:32.697176 (+1672714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.698247 (+  1071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.699807 (+  1560us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54005a55d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:32.700027 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3738889472}
I20260709 13:03:32.706692 17863 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.736321 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 162672us
I20260709 13:03:32.738288 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:32.746112 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2301) took 1666 ms. Trace:
I20260709 13:03:32.746659 15086 rpcz_store.cc:276] 0709 13:03:31.079623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.079778 (+   155us) service_pool.cc:224] Handling call
0709 13:03:31.100120 (+ 20342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.983602 (+883482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.745733 (+762131us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:32.746009 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974139776}
I20260709 13:03:32.758322 17916 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.763317 17895 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:32.766502 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:32.763880 17895 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:32.768064 17826 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:03:32.777019 17895 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.772609 17546 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:32.779006 17895 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:03:32.790665 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 870035us
I20260709 13:03:32.790688 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:32.797020 15203 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.712s (SENT)
I20260709 13:03:32.797493 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:32.799465 15203 leader_election.cc:336] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.923s (SENT)
I20260709 13:03:32.802168 17916 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:32.798439 17816 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:32.811491 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.811872 17816 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.813599 15081 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:32.814411 17592 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:32.815017 17592 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:32.815948 17592 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.779371 17546 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.764453 17834 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:03:32.817106 17546 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:32.817598 17834 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:32.817966 17834 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:32.818790 17834 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.819957 17834 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:32.828060 17908 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:32.789628 17780 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:32.836344 15085 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:32.845018 17780 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:32.846499 14982 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:32.847505 17780 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:32.798539 17761 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:03:32.849956 17761 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:32.848299 14982 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.850322 17761 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:32.795034 17880 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:32.799036 17787 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:03:32.814579 17816 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.742084 17863 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.856869 17863 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.858625 17761 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.860006 17761 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:32.787832 17823 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:32.867679 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 538) took 3427 ms. Trace:
I20260709 13:03:32.868117 15161 rpcz_store.cc:276] 0709 13:03:29.440530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.441317 (+   787us) service_pool.cc:224] Handling call
0709 13:03:29.441687 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.002815 (+1561128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.867445 (+1864630us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540037e954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:32.867619 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3507039488}
I20260709 13:03:32.870330 17546 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.803594 17936 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:32.865267 17900 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:32.853193 17791 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.880396 17925 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:32.873150 17936 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:32.889029 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 133045us
I20260709 13:03:32.875974 17863 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.881948 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81323us
W20260709 13:03:32.901530 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.904318 15203 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.959s (SENT)
I20260709 13:03:32.904796 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:32.906798 15203 leader_election.cc:336] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.292s (SENT)
W20260709 13:03:32.902382 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.763s (SENT)
I20260709 13:03:32.907683 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:32.908023 15128 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.792s (SENT)
I20260709 13:03:32.909261 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:32.909615 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:32.910162 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.910461 15158 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:03:32.908429 15159 raft_consensus.cc:2376] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:32.881291 17546 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:32.910818 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:32.914469 15157 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:32.912549 17781 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.928416 15163 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:32.929495 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:32.930573 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:32.931126 15163 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:32.933315 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.935458 15155 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:32.939453 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:32.912209 17791 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:32.941305 15163 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:32.941764 15159 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:32.943830 15157 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:32.944311 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:32.940533 17619 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:32.946579 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.947496 17895 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:32.913879 17929 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.955694 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:32.961275 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.963698 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:32.964572 15080 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:32.964887 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.965708 15085 raft_consensus.cc:2368] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7.
I20260709 13:03:32.912829 17863 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:32.967911 15203 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 3.654s (SENT)
I20260709 13:03:32.968669 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:32.941937 17880 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:32.952948 17834 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.962687 17895 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.951898 17619 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:32.977140 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:32.978152 15079 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:32.976042 17900 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:32.978868 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:32.973363 17787 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:32.996469 17787 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:32.969602 17893 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.954104 17929 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:32.967412 17819 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:32.967243 17603 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.967082 17925 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:03:32.981957 17936 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:32.983579 17863 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:32.998229 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:32.976291 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.998939 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 310806us
I20260709 13:03:33.007954 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:32.973037 17867 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.013312 17834 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:33.014847 17867 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.015170 17834 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:32.996971 17787 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.017228 17867 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.022219 17592 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:33.022935 17592 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.000474 17619 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.027078 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.029106 17929 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.028949 17819 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.011264 15087 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:03:33.031105 17819 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.031491 15839 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:33.032001 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.013540 17546 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:32.997462 17895 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.016630 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 561) took 1046 ms. Trace:
I20260709 13:03:33.034299 17897 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.035737 17895 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.009326 17603 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.029245 15158 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:33.032366 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.034209 15229 rpcz_store.cc:276] 0709 13:03:31.969854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.970019 (+   165us) service_pool.cc:224] Handling call
0709 13:03:32.005337 (+ 35318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.082233 (+ 76896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.082263 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.098319 (+ 16056us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.577864 (+479545us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.577904 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.016265 (+438361us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5400276fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:33.016313 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.016357 (+    44us) raft_consensus.cc:1685] finished
0709 13:03:33.016370 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.016568 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95330,"raft.queue_time_us":406175,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":115190,"spinlock_wait_cycles":24169856,"thread_start_us":429510,"threads_started":1,"wal-append.queue_time_us":472881}
I20260709 13:03:33.039541 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.028482 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.040546 15155 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:03:33.007367 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.041234 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.060088 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.060750 15162 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:33.061178 17603 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.061630 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.062824 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:33.063114 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 549) took 3451 ms (client timeout 1970 ms). Trace:
I20260709 13:03:33.063320 15162 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:03:33.063545 15235 rpcz_store.cc:269] 0709 13:03:29.611701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.634888 (+ 23187us) service_pool.cc:224] Handling call
0709 13:03:33.062934 (+3428046us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b5400294d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:33.063051 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150785408}
I20260709 13:03:33.065150 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.065263 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.066449 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.066581 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1842) took 1180 ms. Trace:
I20260709 13:03:33.016510 17834 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.066964 15228 rpcz_store.cc:276] 0709 13:03:31.886288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.886536 (+   248us) service_pool.cc:224] Handling call
0709 13:03:31.893246 (+  6710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.674720 (+781474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.066375 (+391655us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b5400318b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:33.066527 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762905856}
I20260709 13:03:33.065443 17798 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.077598 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:33.078207 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.078313 15843 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:33.079456 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.079572 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:33.080322 15082 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:33.081106 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.081233 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.081831 15082 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:33.082316 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.083137 15844 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:03:33.027099 17619 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:33.083940 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.084532 15844 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
W20260709 13:03:33.045817 17921 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.682s	user 0.004s	sys 0.000s
I20260709 13:03:33.097697 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.097896 15162 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:03:33.068323 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 843) took 3370 ms (client timeout 2033 ms). Trace:
W20260709 13:03:33.098603 15845 rpcz_store.cc:269] 0709 13:03:29.698082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.698852 (+   770us) service_pool.cc:224] Handling call
0709 13:03:33.068142 (+3369290us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:33.068258 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4841316224}
I20260709 13:03:33.099022 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.099172 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.067001 15084 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:33.099608 15844 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.063107 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:33.100490 15079 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:33.067639 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:33.099920 15162 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
W20260709 13:03:33.071455 15052 connection.cc:256] RPC call timeout handler was delayed by -1.1261903219999567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 13:03:33.101642 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.101836 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.102213 15162 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:33.103015 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.103483 15162 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.104909 17844 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.107779 17704 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.085098 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.066846 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.110512 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.111352 15846 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:33.085697 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.112895 15842 raft_consensus.cc:2410] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:33.089123 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.085117 17592 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:33.090874 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.114995 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.119259 15229 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:33.067323 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:33.120604 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.070942 17901 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:33.093498 17871 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.071385 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.122087 15087 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:33.080749 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.097994 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.126569 17863 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:33.127005 17863 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.127363 17863 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.067225 17899 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:33.129109 17899 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.129608 17899 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.099886 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.131748 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.133074 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.133770 15845 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:33.134567 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:33.134829 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.135458 15845 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:33.073444 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.137036 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.071468 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.138695 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.139523 15161 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:33.115461 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:33.066985 15054 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.201s (SENT)
I20260709 13:03:33.116377 15079 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:33.141412 15155 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:33.045578 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.142370 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.143081 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.143548 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.144114 15155 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.144388 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.123986 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.145670 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:33.124596 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57806.4us
I20260709 13:03:33.147341 15840 raft_consensus.cc:2368] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4.
I20260709 13:03:33.148339 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.140676 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.149554 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.149911 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:33.150614 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.150472 15161 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:33.153116 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 86156us
I20260709 13:03:33.153191 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.163323 15845 raft_consensus.cc:2410] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:33.169569 15162 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:33.171592 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56660.6us
I20260709 13:03:33.174516 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.175595 15235 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:33.182511 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:33.188527 15845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:03:33.189435 15845 raft_consensus.cc:606] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.189936 15845 raft_consensus.cc:614] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:33.044863 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.004787 17893 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.198161 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.198868 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.124782 17704 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.208671 15228 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:03:33.217677 15235 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:33.219136 15155 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:33.225869 15844 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:33.210513 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.172497 17780 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:33.024613 17787 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.240285 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.232906 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.242030 15155 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:33.172313 17603 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:33.181934 17668 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:03:33.250761 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.251586 15229 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:33.252462 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.252817 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.253145 15236 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:33.253243 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.253605 15231 raft_consensus.cc:2410] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:03:33.254079 15228 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:33.254576 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.255118 15155 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.236820 15844 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:33.242424 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.197643 17819 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.261345 15229 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:33.262792 15235 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:33.256511 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:33.197685 17844 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:03:33.212199 17867 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:03:33.171411 17901 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.197978 17824 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:03:33.279135 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 811) took 1031 ms. Trace:
I20260709 13:03:33.266718 15842 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:33.268656 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.270241 15234 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:33.281374 15230 raft_consensus.cc:2410] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:03:33.268170 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.271879 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.272214 17902 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.273176 17914 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.263398 15839 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:33.279579 15847 rpcz_store.cc:276] 0709 13:03:32.247948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.248094 (+   146us) service_pool.cc:224] Handling call
0709 13:03:33.277832 (+1029738us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b540036db54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:33.279072 (+  1240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651779456}
I20260709 13:03:33.292699 15845 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 7. Ops: []
I20260709 13:03:33.292809 15844 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEARNER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:03:33.283211 15231 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:33.232268 17768 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:03:33.255386 17780 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:33.295646 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.296592 15230 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
W20260709 13:03:33.311859 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.312626 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.322082 15847 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:03:33.322613 15847 pending_rounds.cc:85] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 5
I20260709 13:03:33.322908 15847 pending_rounds.cc:107] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:03:33.323230 15847 raft_consensus.cc:2889] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.326140 15844 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:33.285135 15155 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:33.285564 15162 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
W20260709 13:03:33.290923 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136776us
I20260709 13:03:33.337417 15842 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:33.277941 17819 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:33.351866 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.355123 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:33.356242 15230 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:33.319242 17864 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:33.356298 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.319190 17867 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:33.364965 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.367175 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.368343 15155 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:33.364636 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.369999 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.371182 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.371999 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:33.373292 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 200208us
I20260709 13:03:33.373932 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:33.376505 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:33.377315 15085 raft_consensus.cc:2376] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:33.377454 15084 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:33.376475 15231 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:33.378417 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.359316 15843 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:03:33.316812 17703 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:33.403736 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:33.404596 15081 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.364126 17891 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:33.392810 15230 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:33.396114 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.362275 17864 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:33.363691 17867 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:33.386408 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.414516 15162 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:33.418170 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:33.419047 15231 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.421367 15085 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:33.372486 17896 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:33.376233 17913 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:33.357961 17902 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:03:33.411363 17823 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:33.376159 17733 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:33.360532 17603 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:33.463380 17863 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.428278 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.471774 15155 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:33.443802 15052 leader_election.cc:400] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4.
I20260709 13:03:33.425115 17959 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:33.425164 17780 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.455535 17893 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.456208 17787 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:33.439958 17882 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:33.461162 17913 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:33.462172 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.463008 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.425869 17768 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:33.464592 17603 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.518848 17603 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.431604 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.465380 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.467720 15231 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:33.488802 17959 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.473366 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.389820 17824 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:33.483819 15161 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:33.426991 17896 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:33.496273 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 299717us
W20260709 13:03:33.461258 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.486195 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.440338 17891 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:33.505966 15234 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:33.469944 17897 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:33.519572 17912 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.847s	user 0.016s	sys 0.014s
I20260709 13:03:33.473682 15052 leader_election.cc:403] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:33.432091 17899 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.521283 15230 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:33.521692 15229 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:33.525233 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2297) took 2908 ms. Trace:
I20260709 13:03:33.552552 15083 rpcz_store.cc:276] 0709 13:03:30.616994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.619536 (+  2542us) service_pool.cc:224] Handling call
0709 13:03:30.619830 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.572260 (+1952430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.524895 (+952635us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:33.525151 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4463927168}
I20260709 13:03:33.536082 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.554569 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:33.539074 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.544010 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:33.543877 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57827.4us
I20260709 13:03:33.558645 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.528446 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.547194 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.559887 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.517303 15081 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:33.561707 15162 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:33.523263 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.562991 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.563865 15162 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:33.534368 17959 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.545665 17896 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.493501 17823 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.566198 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:33.570011 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126382us
I20260709 13:03:33.588694 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.589133 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.594930 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.593214 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:33.562244 17824 consensus_queue.cc:1243] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:33.545630 17897 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:33.623088 17897 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:33.629426 15156 raft_consensus.cc:2368] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:33.596144 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.589921 15162 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
W20260709 13:03:33.600595 15203 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 4.132s (SENT)
I20260709 13:03:33.630470 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.630991 17866 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:33.631548 17866 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:33.634366 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.548339 17893 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:33.606516 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.602939 17960 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:03:33.609086 17891 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.610304 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:33.610616 17953 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:33.617154 17863 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.618876 15086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:33.595530 17913 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.621305 17603 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:33.621820 17954 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:03:33.710841 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.581634 17785 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.622185 17895 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:33.622578 17950 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:03:33.584733 17900 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:33.629186 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:33.582425 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136677us
I20260709 13:03:33.738524 15088 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:33.570092 17914 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:33.561116 17768 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:33.745740 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:33.746392 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.602939 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:33.631099 15162 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.748620 15162 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:33.748988 15162 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:33.750160 15162 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.751444 15162 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.752472 17907 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.699414 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.704202 15086 raft_consensus.cc:606] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.588830 17733 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4.
I20260709 13:03:33.598490 17882 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:33.631031 15156 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:33.635722 17798 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.737941 17900 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:33.741834 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 197881us
I20260709 13:03:33.746800 17631 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:33.571499 17598 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:33.773573 17900 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:33.760471 17882 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.762156 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.763084 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.720794 17603 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:33.719615 17895 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.781903 17603 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:33.782159 17895 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:33.782933 17906 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 2.506s	user 0.012s	sys 0.018s
I20260709 13:03:33.782883 17603 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.784179 17603 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.632507 17866 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.785300 17866 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.785192 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.768126 15155 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:33.748159 17704 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.002s
I20260709 13:03:33.740685 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:33.768725 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 229731us
I20260709 13:03:33.740831 17914 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.791610 17914 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:33.772665 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.774135 17798 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.797780 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:33.798914 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.799115 17961 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:33.799460 17798 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:33.735236 17785 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.799785 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:33.800892 15154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1851) took 3997 ms (client timeout 1804 ms). Trace:
I20260709 13:03:33.801074 17929 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:33.801718 17929 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.801838 17785 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:33.755981 17893 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:33.802191 17929 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.803577 17899 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:33.803926 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.693128 15158 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:33.807734 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.716986 17863 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:33.809434 17863 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:33.809731 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.809998 17863 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:33.811462 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.812124 17768 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.815897 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:33.816927 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:33.817626 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.818401 15235 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:33.786741 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.748919 15161 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:33.817596 15088 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:03:33.801353 15154 rpcz_store.cc:269] 0709 13:03:29.803612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.803745 (+   133us) service_pool.cc:224] Handling call
0709 13:03:33.800649 (+3996904us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b54001d8654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:33.800790 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6698691840}
I20260709 13:03:33.821238 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:33.821928 15234 raft_consensus.cc:2376] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:33.821766 17959 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.823783 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:33.824312 17959 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:33.824555 15230 raft_consensus.cc:2376] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4. Re-sending same reply.
I20260709 13:03:33.827109 15088 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:33.801869 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 550) took 4108 ms. Trace:
I20260709 13:03:33.804592 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:33.811188 17863 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.832576 15162 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:33.838902 15161 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:33.839564 15162 pending_rounds.cc:85] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 6
I20260709 13:03:33.799685 17961 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.837618 15230 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:33.837889 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.773780 17631 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:33.839923 15162 pending_rounds.cc:107] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:33.928148 15162 raft_consensus.cc:2889] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.928670 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.901276 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:33.858307 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 224200us
I20260709 13:03:33.893188 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.930683 15087 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 2.
I20260709 13:03:33.899637 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:33.931900 15844 raft_consensus.cc:2410] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:33.812145 15843 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.828091 15237 rpcz_store.cc:276] 0709 13:03:29.693975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.675678 (+981703us) service_pool.cc:224] Handling call
0709 13:03:30.726884 (+ 51206us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.012578 (+285694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.012613 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.021859 (+  9246us) log.cc:844] Serialized 52 byte log entry
0709 13:03:31.074405 (+ 52546us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:31.074452 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.074528 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.740148 (+2665620us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:33.801551 (+ 61403us) raft_consensus.cc:1685] finished
0709 13:03:33.801574 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.801803 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36270,"raft.queue_time_us":264650,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7315,"spinlock_wait_cycles":34580864,"thread_start_us":241710,"threads_started":2,"wal-append.queue_time_us":226830}
W20260709 13:03:33.934859 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 334288us
I20260709 13:03:33.891337 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:33.878211 17961 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:33.826022 17871 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.005s
I20260709 13:03:33.781105 17882 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:33.826225 17893 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.829502 17598 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.833272 17798 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.920634 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:33.926932 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.929949 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.846669 17704 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:33.935191 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:33.936030 15203 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 4.240s (SENT)
I20260709 13:03:33.973443 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:33.974067 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:33.974356 15847 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:33.891531 15234 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:33.929844 15159 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:33.902856 15229 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:33.899547 15841 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:33.939098 15848 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:33.900588 17885 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:33.946143 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 352996us
W20260709 13:03:33.978657 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48801.8us
I20260709 13:03:33.979519 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:33.979803 15160 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:03:33.980274 15160 pending_rounds.cc:85] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 2
I20260709 13:03:33.980473 15160 pending_rounds.cc:107] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:03:33.980726 15160 raft_consensus.cc:2889] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.982414 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.954778 17929 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.956305 17631 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.962265 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 565) took 1712 ms. Trace:
I20260709 13:03:33.985369 15232 rpcz_store.cc:276] 0709 13:03:32.249720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.251185 (+  1465us) service_pool.cc:224] Handling call
0709 13:03:32.251516 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.673520 (+422004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.673548 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.673588 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.673624 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.961922 (+1288298us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54000591d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:33.961994 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.962178 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970083584}
I20260709 13:03:33.982790 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:33.986944 17871 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:33.982304 17798 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.988617 17864 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:33.990015 17864 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:33.990597 17798 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:33.991853 17977 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:33.994561 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:33.996626 15234 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:33.998198 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:33.999783 15232 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:33.979357 17546 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.973477 17704 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.992470 17980 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:03:33.978950 17885 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:34.016251 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.026235 17885 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:34.017012 17631 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:33.984154 17780 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:03:34.008903 17798 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:33.997742 17899 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:03:33.982470 17897 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:33.979094 17893 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.023908 17704 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:34.080685 15842 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:34.015816 17976 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.009673 17703 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:34.090678 15848 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:34.046653 17598 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:34.023934 17546 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.135926 15084 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:34.138842 15231 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:34.124758 17884 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:34.145301 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 147173us
I20260709 13:03:34.147908 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:34.133366 17598 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:34.135144 17798 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:34.141995 17884 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:34.169476 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 190039us
I20260709 13:03:34.173288 15847 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:34.177135 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 256615us
I20260709 13:03:34.178294 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:34.177937 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:34.168597 17986 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.173686 17785 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:34.157660 17980 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:03:34.201367 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.214113 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46593.5us
W20260709 13:03:34.220005 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.191896 17929 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.192341 17977 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:34.222100 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 286105us
I20260709 13:03:34.228197 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:34.228830 15230 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:34.234084 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:34.234905 15233 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:34.225193 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46986us
I20260709 13:03:34.236562 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:34.242044 15230 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:34.243484 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:34.244071 15230 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:34.172978 17598 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.248512 17785 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:34.250618 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49357.5us
I20260709 13:03:34.252641 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.254489 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:34.247694 17896 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.006s
I20260709 13:03:34.202219 17993 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:03:34.215463 17914 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.259338 15236 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:34.203624 17986 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.274202 15081 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:34.258385 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.259521 17979 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:03:34.264088 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 825) took 1061 ms. Trace:
W20260709 13:03:34.269714 17985 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:03:34.271112 15230 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:34.254140 17991 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.273581 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:34.275825 17957 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.774s	user 0.001s	sys 0.000s
W20260709 13:03:34.293457 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.302764 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.312743 15234 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:34.262434 17893 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:34.314183 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55933.9us
I20260709 13:03:34.315209 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:34.302196 15233 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:34.304096 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:34.298722 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 457513us
W20260709 13:03:34.290226 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.255770 17996 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.309561 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.263170 17929 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.319337 15230 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:34.319471 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.318177 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.273200 18002 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:34.320055 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83601.8us
I20260709 13:03:34.273698 17914 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:34.311092 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.322641 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.277457 17704 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.354664 15229 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 2.
I20260709 13:03:34.355705 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.356164 17704 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:34.323479 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.356950 17785 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.357934 15843 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:34.323529 17986 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:34.318156 17991 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.313406 17976 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:34.361388 17976 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.327569 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.329782 17900 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.001s
I20260709 13:03:34.305071 15839 rpcz_store.cc:276] 0709 13:03:33.199539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.355329 (+155790us) service_pool.cc:224] Handling call
0709 13:03:33.389065 (+ 33736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.430628 (+ 41563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.435111 (+  4483us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.545971 (+110860us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.868957 (+322986us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:33.902755 (+ 33798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.912111 (+  9356us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.250598 (+338487us) raft_consensus.cc:1685] finished
0709 13:03:34.250628 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.261282 (+ 10654us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":238591,"threads_started":1,"wal-append.queue_time_us":464684}
I20260709 13:03:34.330356 18002 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:34.331017 15188 heartbeater.cc:646] Failed to heartbeat to 127.13.163.254:45133 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.13.163.254:45133 timed out after 15.000s (SENT)
I20260709 13:03:34.331665 17819 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.332147 17996 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.333459 17929 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:34.404601 15231 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:34.336388 15230 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:34.337903 18006 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:34.290287 18004 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:34.309204 17980 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:34.353580 17992 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:03:34.311884 17984 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:03:34.363317 15847 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:03:34.364125 15087 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:34.377797 17893 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:34.289892 17995 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.366531 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.428097 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61619.2us
I20260709 13:03:34.429155 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:34.430634 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:34.369549 15161 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:34.370224 15158 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:34.371523 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:34.433486 15233 raft_consensus.cc:2368] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4.
I20260709 13:03:34.371762 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:34.434844 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.372433 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44943.2us
I20260709 13:03:34.436496 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:34.437762 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:34.373384 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.378407 17914 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.374720 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:34.377820 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.400502 15088 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:34.348096 17882 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:34.402465 17895 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.002s
I20260709 13:03:34.282410 17896 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.418399 17864 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:34.423692 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:34.377460 17900 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.378099 18001 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:34.374617 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.440436 15080 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:34.442597 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:34.443445 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.447016 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:34.451695 17899 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.452953 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:34.453874 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74498.5us
I20260709 13:03:34.440250 17603 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:34.456883 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.439788 17703 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:34.504259 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:34.425989 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 108261us
I20260709 13:03:34.519343 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:34.512176 15233 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:34.311794 17977 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.515000 15079 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:34.467628 15083 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:34.444062 17844 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:34.517627 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:34.466632 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:34.449265 18004 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:34.448726 17819 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:34.505654 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52838.3us
W20260709 13:03:34.516062 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 73528.8us
I20260709 13:03:34.454151 17914 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.447809 17994 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:03:34.531776 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:34.514197 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 266441us
I20260709 13:03:34.559753 15155 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:34.553745 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:34.530572 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:34.523073 17864 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:34.458959 17895 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.454493 18008 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:03:34.523159 17603 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:34.442338 17995 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.514571 17768 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:34.561928 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:34.566130 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:34.527908 17988 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:03:34.562947 18013 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:34.527477 18011 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:34.582849 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:34.524108 17844 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.588325 18009 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.588663 17901 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.555356 17989 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:34.588986 17900 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:34.554456 18019 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:34.527343 17704 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:03:34.590999 17896 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.593266 17866 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:34.594055 17991 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.553714 17993 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:03:34.594326 15229 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:34.597523 15233 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:34.599560 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80301.9us
I20260709 13:03:34.558600 17819 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:34.656016 17819 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:34.657238 17983 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.514s	user 0.003s	sys 0.001s
I20260709 13:03:34.660915 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34666 (request call id 22) took 1042 ms. Trace:
I20260709 13:03:34.661401 15086 rpcz_store.cc:276] 0709 13:03:33.618449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.618674 (+   225us) service_pool.cc:224] Handling call
0709 13:03:34.660648 (+1041974us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4c00177ec0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3dcb0a20 00007f0d3dcc1a29 00007f0d3dc87644 00007f0d3dcaef2d 00007f0d3dcaeb93 00007f0d3f9dec92 00007f0d3f9de84a 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691
0709 13:03:34.660790 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39429888}
I20260709 13:03:34.662881 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1874) took 1125 ms. Trace:
I20260709 13:03:34.642608 18019 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.667299 18019 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:34.671482 17955 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.130s	user 0.004s	sys 0.000s
I20260709 13:03:34.672986 17603 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.619501 18011 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.603288 15235 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:34.533329 18004 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.602164 17768 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:34.514312 18001 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.619921 15228 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:34.602293 17929 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:34.566632 17986 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:03:34.700218 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:34.644640 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 65908.4us
I20260709 13:03:34.642750 18013 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:34.643134 17901 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.711005 17901 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.711515 18001 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.694680 18011 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:34.713366 17863 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.714010 17863 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.717027 17631 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.717741 17895 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.717592 17631 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.719012 18016 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:34.719580 18016 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:34.719709 17631 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.720466 18016 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.721642 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1852) took 4098 ms. Trace:
I20260709 13:03:34.721840 15086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:34.647347 17882 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:34.648281 17989 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:34.648353 18002 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.649917 17995 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.616694 17958 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:34.653009 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122512us
I20260709 13:03:34.654964 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:34.655611 18012 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:03:34.663307 15156 rpcz_store.cc:276] 0709 13:03:33.537122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.789808 (+252686us) service_pool.cc:224] Handling call
0709 13:03:33.790110 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.790492 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:33.790868 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.013273 (+222405us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.022519 (+  9246us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.022665 (+   146us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:34.025537 (+  2872us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.280081 (+254544us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18002ca640. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:34.365445 (+ 85364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.365513 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.656425 (+290912us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:34.662640 (+  6215us) raft_consensus.cc:1685] finished
0709 13:03:34.662665 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.662811 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34235,"prepare.queue_time_us":12382,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":264883,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":8716,"spinlock_wait_cycles":14173312,"thread_start_us":553715,"threads_started":2,"wal-append.queue_time_us":294524}
I20260709 13:03:34.713493 17901 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:34.698974 17961 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.738s	user 0.003s	sys 0.008s
W20260709 13:03:34.700094 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.640247 17780 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.603189 17703 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:34.622048 17844 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:34.733989 17958 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.708206 17768 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.710098 18017 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.722059 15160 rpcz_store.cc:276] 0709 13:03:30.623434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.637827 (+ 14393us) service_pool.cc:224] Handling call
0709 13:03:30.638127 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.985991 (+3347864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.721425 (+735434us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b540017d754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:34.721591 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7678958336}
W20260709 13:03:34.729243 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 164467us
I20260709 13:03:34.790005 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.790853 15160 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:34.793689 18017 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:34.795404 17885 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.709s	user 0.006s	sys 0.004s
W20260709 13:03:34.734514 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79669.8us
W20260709 13:03:34.763085 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 59364.3us
I20260709 13:03:34.728446 17989 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:34.795938 17885 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:34.797049 18013 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.797847 17631 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.798553 17885 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:34.798745 17631 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.800519 17996 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.802837 17631 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.804545 17631 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.805354 17631 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:34.764407 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.808152 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.808748 15154 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:34.702478 18004 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:34.767292 15157 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:03:34.810009 15157 pending_rounds.cc:85] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 2
I20260709 13:03:34.810281 15157 pending_rounds.cc:107] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:03:34.810663 15157 raft_consensus.cc:2889] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:34.686592 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.778818 15156 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:34.811684 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.812552 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.813124 15156 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:34.813269 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.723847 17863 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:34.794143 17914 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.813916 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.749655 17882 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:34.811684 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.812669 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:34.834559 15157 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6. Re-sending same reply.
I20260709 13:03:34.827819 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.796350 18017 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:34.831281 15155 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:34.759198 17986 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:03:34.758913 17866 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:03:34.779474 17864 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:34.846020 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.846410 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.847157 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.853600 15842 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:34.854714 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.855772 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.777412 17703 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:34.853564 15157 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:34.895169 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.787321 17958 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:34.858920 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.852470 17885 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:03:34.902118 17867 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.903563 17867 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:34.908905 17959 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:34.909580 17631 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:34.849139 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.867187 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:34.833060 17882 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.759785 18019 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:34.851159 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.777297 17961 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:34.889145 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:34.909518 17959 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:34.879130 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 744) took 1505 ms. Trace:
I20260709 13:03:34.855468 17603 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:03:34.871706 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:34.896587 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:35.059116 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 573) took 1227 ms. Trace:
W20260709 13:03:34.977483 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 142543us
I20260709 13:03:34.940752 15154 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:35.169166 15235 rpcz_store.cc:276] 0709 13:03:33.831729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.831849 (+   120us) service_pool.cc:224] Handling call
0709 13:03:33.832143 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.677140 (+844997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.058749 (+381609us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540005cb54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:35.059025 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773055232}
W20260709 13:03:35.097513 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 230412us
I20260709 13:03:35.101321 15079 rpcz_store.cc:276] 0709 13:03:33.374053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.374170 (+   117us) service_pool.cc:224] Handling call
0709 13:03:33.374516 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.521602 (+1147086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.878723 (+357121us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000db6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:34.879042 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622106496}
I20260709 13:03:35.102552 15155 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:35.066942 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.171622 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:34.958916 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 337097us
W20260709 13:03:35.174772 15081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2310) took 1470 ms (client timeout 1783 ms). Trace:
W20260709 13:03:35.175385 15081 rpcz_store.cc:269] 0709 13:03:33.704743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.711984 (+  7241us) service_pool.cc:224] Handling call
0709 13:03:35.174542 (+1462558us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540011add4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:35.174703 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113578368}
I20260709 13:03:34.989634 15236 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.177927 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:35.178900 15842 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:34.897480 15160 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:34.917289 17703 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:35.186852 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:35.194566 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:35.206007 15051 leader_election.cc:400] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4.
I20260709 13:03:35.207705 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:35.194975 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:35.210566 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:35.211700 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:35.212659 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.205099 15845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.020s
W20260709 13:03:35.214792 15845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.004s	sys 0.020s
I20260709 13:03:35.217860 15157 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:35.222074 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:35.222766 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.224347 15157 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.249622 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:35.225492 15156 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:34.897903 18034 raft_consensus.cc:993] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
W20260709 13:03:34.987701 17893 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:03:35.173524 17882 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:35.255425 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:35.341576 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 86220.3us
W20260709 13:03:35.255540 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.256698 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:35.344069 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 87473.6us
I20260709 13:03:35.261739 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.345320 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:35.345431 15155 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:35.193888 18028 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:34.977028 18004 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.224964 17631 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.911666 17844 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.053824 18011 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.380831 18011 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.210386 17780 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:03:35.252557 17592 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:35.258258 17866 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.289705 15157 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:35.256162 18001 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:35.211841 17668 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:35.291445 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:35.211674 17899 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:03:35.266582 17914 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:34.986855 17864 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.268572 18009 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.252699 17900 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:35.258951 17995 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 13:03:35.307930 17768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.020s	sys 0.045s
I20260709 13:03:35.347617 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:35.342943 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.395802 15235 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.348703 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:35.352630 17893 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:35.390579 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 196089us
I20260709 13:03:35.396929 15229 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:35.352312 15228 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:03:35.195998 17798 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:35.312772 17603 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:35.352702 17977 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:35.359828 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:35.360073 17989 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.512s	user 0.006s	sys 0.004s
I20260709 13:03:35.360291 17733 raft_consensus.cc:1081] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:03:35.408713 17977 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.392979 17885 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:03:35.430509 17603 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:35.430717 17977 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.415386 15235 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:35.431581 17603 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.361637 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2309) took 1658 ms. Trace:
I20260709 13:03:35.433063 15084 rpcz_store.cc:276] 0709 13:03:33.703116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.809991 (+106875us) service_pool.cc:224] Handling call
0709 13:03:33.810337 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.136497 (+326160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.361350 (+1224853us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002373d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:35.361569 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748283648}
W20260709 13:03:35.362630 15841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 784) took 2331 ms (client timeout 2191 ms). Trace:
I20260709 13:03:35.363788 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.363662 17958 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:35.364979 18031 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:35.365049 17631 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.348644 17907 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:35.367260 17844 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.369926 17598 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.371562 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.450631 15161 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:35.374202 17961 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:35.376531 18004 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.377219 15162 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:35.059195 17819 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.385969 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.452064 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:35.389950 17914 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.360450 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.464304 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.395478 18009 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.463660 17914 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:35.398521 17768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.020s	sys 0.045s
I20260709 13:03:35.434684 15158 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:35.478029 15158 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:03:35.441399 18019 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:35.434474 15841 rpcz_store.cc:269] 0709 13:03:33.030726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.031750 (+  1024us) service_pool.cc:224] Handling call
0709 13:03:35.362465 (+2330715us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400822154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:35.362566 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741351808}
I20260709 13:03:35.390165 17592 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.505646 15086 raft_consensus.cc:606] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:35.390468 17866 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.408087 17798 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.406075 17880 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:03:35.529279 17880 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:35.529680 17880 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:35.530539 17880 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.531716 17880 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:35.534266 18032 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.622s	user 0.003s	sys 0.005s
W20260709 13:03:35.449000 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 241817us
I20260709 13:03:35.408674 15231 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:35.453094 15157 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:35.439249 17989 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.476049 18004 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:35.466070 17844 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.470315 15155 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:35.540309 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1865) took 1200 ms. Trace:
I20260709 13:03:35.557483 15231 rpcz_store.cc:276] 0709 13:03:34.340345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.555798 (+215453us) service_pool.cc:224] Handling call
0709 13:03:34.574542 (+ 18744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.538045 (+963503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.540077 (+  2032us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400295754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:35.540250 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917840768}
W20260709 13:03:35.547408 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.409252 15237 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 2.
W20260709 13:03:35.472317 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 183908us
I20260709 13:03:35.504429 15236 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:35.514954 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.517421 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.512676 17546 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:35.562212 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44826.5us
I20260709 13:03:35.550598 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:35.566996 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 52135.1us
I20260709 13:03:35.567803 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:35.568889 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.571132 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:35.574909 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.578575 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:35.583678 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.537617 18042 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:03:35.608626 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.637919 15163 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:35.605345 17824 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:35.639300 17824 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.640177 18009 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:35.583489 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:35.584446 15087 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:35.607030 15082 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:35.600158 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:35.553129 17798 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.538971 17819 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:35.646183 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:35.649612 15158 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:35.650272 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 99744.6us
I20260709 13:03:35.652554 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.583022 17896 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.605940 17995 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:35.607447 17893 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:35.607203 17900 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:35.605733 17961 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:35.606025 17907 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:35.583061 18019 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:35.679548 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:35.661784 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:35.698204 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45797.5us
I20260709 13:03:35.698488 15163 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.610268 18001 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:35.605533 18007 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:35.546017 17844 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.706876 15087 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:35.695279 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 345984us
I20260709 13:03:35.545732 18004 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.580536 17882 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.710717 15087 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:35.706502 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.583040 17768 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:03:35.654723 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:35.659376 17668 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:35.536809 17703 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:35.719504 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.719290 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 119205us
I20260709 13:03:35.717496 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.725380 15087 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:35.726436 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.558987 17977 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.682859 17995 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:35.738744 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:35.740007 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:35.730041 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:35.693779 15158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:03:35.741223 15158 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:35.741631 15158 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:35.549114 17959 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.700768 17961 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.700487 17907 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.743127 15080 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:35.702386 17900 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:35.744867 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 867) took 1849 ms. Trace:
I20260709 13:03:35.710224 17891 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:35.749353 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:35.746373 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.761744 15235 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:35.763599 15082 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:35.755997 15230 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:35.758252 15229 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:35.753217 15845 rpcz_store.cc:276] 0709 13:03:33.895710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.895843 (+   133us) service_pool.cc:224] Handling call
0709 13:03:33.896144 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.896425 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:33.910932 (+ 14507us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.670510 (+759578us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18000eb3c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:03:35.744104 (+1073594us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.744416 (+   312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:35.744489 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:35.744548 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.744631 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.744773 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44254,"prepare.queue_time_us":300469,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":2517,"raft.queue_time_us":54966,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":4528512,"thread_start_us":1029146,"threads_started":2,"wal-append.queue_time_us":1363858}
I20260709 13:03:35.695043 17546 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:03:35.707015 17893 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:35.752092 17961 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.782632 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.783094 17893 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.783979 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.784686 15231 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:35.729246 17914 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.785800 15232 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:35.695443 17986 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:03:35.782644 17959 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.786813 17914 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.762655 17891 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:35.787592 17891 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.767685 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.789047 15237 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:35.789131 15231 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:35.790225 15237 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:35.770197 15235 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:35.771724 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.791386 15229 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:35.693419 17896 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.720178 18007 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.799288 17896 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.799762 18043 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:35.801002 17631 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:35.802587 17844 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:35.803977 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.759277 18036 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:03:35.754035 18019 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:03:35.773280 15234 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:35.730554 18001 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:35.754772 18011 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:35.808756 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.744819 17995 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.754427 17907 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.761204 17900 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:35.873332 15230 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:35.807245 15163 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:35.818488 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:35.880432 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 62025.7us
I20260709 13:03:35.880831 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.755681 17824 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.881707 15163 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:35.800750 18043 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:35.804477 17989 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:35.860771 18001 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:35.890080 18001 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.890143 18043 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.891436 18043 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.891618 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:35.799794 18051 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:35.859005 15158 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.821372 15229 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:35.869346 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:35.894234 15234 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:35.892421 15084 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:35.804687 17896 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:35.887049 18054 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:03:35.814738 17914 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:35.900496 15158 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:35.902452 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 172501us
I20260709 13:03:35.906018 15230 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:35.916051 15234 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:35.904243 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.921164 15084 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:35.882743 18044 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:35.897341 18011 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:35.932157 18014 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.482s	user 0.006s	sys 0.007s
I20260709 13:03:35.892392 18001 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.897185 17824 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.896421 18051 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:35.941179 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:35.943562 18019 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:35.923660 17896 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:35.945531 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:35.946973 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.948562 18051 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:35.953202 17977 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:35.955843 17598 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:03:35.957023 15157 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:35.958775 17896 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.959542 17897 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.950631 18019 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:35.961355 17958 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.962020 17703 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:35.962834 17981 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.975s	user 0.000s	sys 0.027s
I20260709 13:03:35.963078 18019 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:35.927640 17914 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.965018 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1886) took 1197 ms. Trace:
I20260709 13:03:35.965376 15156 rpcz_store.cc:276] 0709 13:03:34.767276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.813712 (+ 46436us) service_pool.cc:224] Handling call
0709 13:03:35.964817 (+1151105us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:35.964963 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902288256}
I20260709 13:03:35.966446 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.967008 15156 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:35.967185 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2307) took 2309 ms. Trace:
I20260709 13:03:35.967778 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.968693 15156 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:35.973165 15084 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:35.967730 15085 rpcz_store.cc:276] 0709 13:03:33.658224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.697629 (+ 39405us) service_pool.cc:224] Handling call
0709 13:03:33.713793 (+ 16164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.717447 (+  3654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.717492 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.721151 (+  3659us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.976266 (+255115us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:34.013449 (+ 37183us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.013736 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.013831 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.665251 (+651420us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b18003b4ee0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:34.987064 (+321813us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b18003b4ee0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:35.962266 (+975202us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:35.966943 (+  4677us) raft_consensus.cc:1685] finished
0709 13:03:35.966972 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.967124 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4760,"prepare.queue_time_us":38401,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":9059,"raft.queue_time_us":6118,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":8321,"spinlock_wait_cycles":143483904,"thread_start_us":203221,"threads_started":1,"wal-append.queue_time_us":286560}
I20260709 13:03:35.987521 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.990111 15156 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:35.988572 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:35.991089 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:35.938695 17668 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:35.991091 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:36.010728 15084 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:36.012281 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 102959us
I20260709 13:03:36.016662 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:36.023288 15156 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:36.030685 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.034204 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:36.034312 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:35.935204 18044 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.036339 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:35.989256 17897 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:36.062520 15234 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:36.062757 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.063556 15085 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:03:36.063637 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 111730us
I20260709 13:03:36.038096 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:36.067101 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.060446 15156 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:36.039710 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:36.078336 15814 connection.cc:256] RPC call timeout handler was delayed by -1.2906883750000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3057s for a response
I20260709 13:03:36.080657 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:36.086017 15844 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:36.086582 15844 pending_rounds.cc:85] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 6
I20260709 13:03:36.086865 15844 pending_rounds.cc:107] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:36.087173 15844 raft_consensus.cc:2889] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:36.054837 18016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.006s	sys 0.033s
I20260709 13:03:36.099089 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:36.099918 15846 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:35.964169 17824 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.094902 15845 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:36.094303 18004 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:35.987310 17914 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.106783 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:36.108170 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:36.033468 17976 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.128000 17976 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.131150 17976 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.015215 18037 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:03:36.034785 18056 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:03:36.143087 18044 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.145637 17787 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:36.146445 17787 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:36.147710 17787 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:36.154173 18063 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:36.165385 17668 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.065932 18010 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.109457 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:36.125523 17914 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:36.126859 18058 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:03:36.172894 17884 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.175536 17882 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.178114 17592 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.181438 17866 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.185109 18010 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.189775 18007 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.197654 17961 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.070302 17989 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.097548 18016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.007s	sys 0.033s
I20260709 13:03:36.094465 17986 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.140117 17907 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.097765 18065 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.170s
I20260709 13:03:36.247853 17961 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.188402 18010 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.258374 18051 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.260309 18051 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:36.293301 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:36.296365 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.296578 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:36.297849 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 306675us
I20260709 13:03:36.298501 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:36.299266 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 316016us
I20260709 13:03:36.299420 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:36.300091 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.300877 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:36.301532 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.311851 15084 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:03:36.323959 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 285783us
W20260709 13:03:36.325482 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.332867 15155 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:03:36.336140 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.153s (SENT)
I20260709 13:03:36.341037 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:36.341711 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 307455us
I20260709 13:03:36.114799 15841 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:36.362974 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63822.3us
W20260709 13:03:36.363857 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 350645us
W20260709 13:03:36.371382 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.372265 15234 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:36.113418 18009 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 13:03:36.388778 15156 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:36.392988 15088 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:36.205348 17989 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.405771 17907 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:36.407033 15847 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:03:36.407894 17989 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.408907 18011 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.409967 18011 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.407783 18068 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:36.407786 15847 pending_rounds.cc:85] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 6
I20260709 13:03:36.425450 14982 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 3, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.437009 17871 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:36.440151 17824 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.114754 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:36.072465 17798 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.477979 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34666 (request call id 23) took 1756 ms. Trace:
I20260709 13:03:36.468552 15847 pending_rounds.cc:107] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:03:36.495517 15847 raft_consensus.cc:2889] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:36.498538 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 162531us
I20260709 13:03:36.502874 17780 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:36.503840 17780 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:36.503979 17882 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:36.505481 17882 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:36.505846 17882 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.506224 17882 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.509356 17885 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:36.486361 18065 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.510511 17896 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:36.510983 17896 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:36.511116 18065 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.487769 17598 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:36.514137 18065 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:36.508821 15163 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:36.517809 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.518996 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.493916 15086 rpcz_store.cc:276] 0709 13:03:34.721480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.721721 (+   241us) service_pool.cc:224] Handling call
0709 13:03:36.477482 (+1755761us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:36.477691 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801133952}
I20260709 13:03:36.524017 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1884) took 1757 ms. Trace:
W20260709 13:03:36.525776 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.526036 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 566) took 2156 ms. Trace:
I20260709 13:03:36.526394 15162 rpcz_store.cc:276] 0709 13:03:34.369768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.519962 (+150194us) service_pool.cc:224] Handling call
0709 13:03:34.520229 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.451323 (+931094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.525440 (+1074117us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540017e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:36.525853 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970158336}
W20260709 13:03:36.530494 17995 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.004s
I20260709 13:03:36.530157 17780 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:36.531188 17995 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.535996 17798 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.537344 18061 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:36.540150 18061 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:36.551182 18061 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.554436 15228 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:36.492753 18011 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:36.555933 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 1854) took 2320 ms (client timeout 1972 ms). Trace:
W20260709 13:03:36.556455 15236 rpcz_store.cc:269] 0709 13:03:34.235318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.273283 (+ 37965us) service_pool.cc:224] Handling call
0709 13:03:36.555563 (+2282280us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b5400318654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:36.555722 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565791488}
I20260709 13:03:36.252889 17986 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.512434 17896 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.558156 17995 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.559257 17896 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:36.559895 17986 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:36.564669 17864 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.565579 17823 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.565527 17864 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.566303 17823 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.567062 17823 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.502959 17824 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.569409 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.570482 15231 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:36.570082 17824 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.571975 15231 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:36.524502 15161 rpcz_store.cc:276] 0709 13:03:34.766824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.467289 (+700465us) service_pool.cc:224] Handling call
0709 13:03:35.549358 (+ 82069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.549565 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.549594 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:35.549637 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:35.549680 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.549736 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.523606 (+973870us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54003e0650. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:36.523809 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176003328}
I20260709 13:03:36.581884 17864 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.592504 15233 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:36.592641 18083 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.593556 18083 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.572676 17824 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.595994 17631 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.602226 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 301729us
W20260709 13:03:36.596042 17891 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.005s
I20260709 13:03:36.593556 18010 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.610292 17823 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.613142 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:36.604250 17631 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:36.614076 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 763) took 1135 ms. Trace:
W20260709 13:03:36.614190 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 126851us
W20260709 13:03:36.616482 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55569.7us
I20260709 13:03:36.608279 17891 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.617595 15231 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:36.653043 18010 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.654698 15231 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:36.656615 15231 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:36.649243 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:36.628453 17900 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.004s
I20260709 13:03:36.629662 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:36.664477 18061 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:36.675753 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:36.664426 18010 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.689198 18056 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.681587 18083 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.689972 18056 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:36.823976 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.729442 15080 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:36.790934 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105571us
W20260709 13:03:36.735905 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52700.9us
W20260709 13:03:36.800544 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 86531.6us
I20260709 13:03:36.724730 15237 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:36.731739 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.820554 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:36.830853 15231 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:36.731403 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 590133us
I20260709 13:03:36.660321 15083 rpcz_store.cc:276] 0709 13:03:35.469600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.469787 (+   187us) service_pool.cc:224] Handling call
0709 13:03:35.470210 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.473034 (+  2824us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:35.751179 (+278145us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.387413 (+636234us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.548609 (+161196us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.569150 (+ 20541us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:36.569203 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.569349 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.596525 (+ 27176us) raft_consensus.cc:1685] finished
0709 13:03:36.596545 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.604677 (+  8132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":476,"prepare.queue_time_us":316068,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":22,"raft.queue_time_us":280565,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":358,"thread_start_us":668700,"threads_started":3,"wal-append.queue_time_us":279774}
W20260709 13:03:36.774258 15202 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.783s (SENT)
I20260709 13:03:36.851910 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.853826 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.854998 17880 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:36.855370 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:36.856369 15230 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:36.856014 17880 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.860272 17986 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:36.731159 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:36.731359 17891 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:36.865792 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:36.866566 17986 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.681989 17900 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.794489 18023 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:36.799135 17899 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:03:36.879662 15233 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:36.881824 17986 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.882565 17986 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:36.874650 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:36.883656 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.877427 18010 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:36.890271 18009 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:36.890694 15237 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:36.891340 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.892094 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.896795 15232 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:36.909173 18016 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:03:36.911149 17880 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.877208 18023 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:36.914072 17631 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:36.913832 18023 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.915109 18023 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:36.915675 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 762) took 1453 ms. Trace:
I20260709 13:03:36.916059 15081 rpcz_store.cc:276] 0709 13:03:35.462557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.465014 (+  2457us) service_pool.cc:224] Handling call
0709 13:03:35.472405 (+  7391us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:35.730790 (+258385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.730821 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:36.440238 (+709417us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b2c000ecb98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:36.477826 (+ 37588us) log.cc:844] Serialized 78 byte log entry
0709 13:03:36.572858 (+ 95032us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:36.573212 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.573480 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.573613 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.915243 (+341630us) raft_consensus.cc:1685] finished
0709 13:03:36.915337 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.915512 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":409,"prepare.queue_time_us":1542,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2983,"raft.queue_time_us":92972,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":218,"spinlock_wait_cycles":49481984,"thread_start_us":259258,"threads_started":2,"wal-append.queue_time_us":367095}
I20260709 13:03:36.918455 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:36.912668 17880 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:36.897682 18010 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:36.930884 15233 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:36.938895 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:36.930917 17882 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.941656 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:36.942934 17882 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:36.941947 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 82586.7us
W20260709 13:03:36.950666 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 176504us
W20260709 13:03:36.952548 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 255764us
W20260709 13:03:36.953527 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 95343.8us
I20260709 13:03:36.959014 15087 raft_consensus.cc:1217] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-6.7]   Dedup: 6.7->[]
I20260709 13:03:36.959793 15087 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:03:36.960448 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:36.962852 15081 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:36.966805 17899 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:36.968266 17899 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:36.932618 17880 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:36.969748 17831 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:36.970311 17831 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:36.971215 17631 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.971519 17831 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.973266 17831 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:36.973856 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:36.974486 15154 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:36.971976 17631 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.977391 17899 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:36.979324 17899 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:36.987373 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1703) took 1288 ms. Trace:
I20260709 13:03:36.988013 15088 rpcz_store.cc:276] 0709 13:03:35.699300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.699912 (+   612us) service_pool.cc:224] Handling call
0709 13:03:36.987007 (+1287095us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b5400190354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:36.987212 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1373041920}
W20260709 13:03:36.980703 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.981244 18009 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:36.989281 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 775) took 1111 ms. Trace:
I20260709 13:03:36.990713 15083 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:36.939837 18010 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:36.992116 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:37.001241 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67111.6us
I20260709 13:03:36.980309 17631 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.005669 18009 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:37.017171 17668 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:37.018110 17668 consensus_queue.cc:1243] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:37.021365 17900 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.021839 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.023715 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.027257 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.028266 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.028762 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:37.029264 15081 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:37.030891 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:36.995952 15086 rpcz_store.cc:276] 0709 13:03:35.878293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.525106 (+646813us) service_pool.cc:224] Handling call
0709 13:03:36.525745 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.617788 (+ 92043us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:36.617818 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.618470 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.680085 (+ 61615us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:36.680128 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.680211 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.988934 (+308723us) raft_consensus.cc:1685] finished
0709 13:03:36.988972 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.989203 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33975,"raft.queue_time_us":63300,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":61088,"threads_started":1,"wal-append.queue_time_us":366390}
I20260709 13:03:37.033676 17896 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:03:37.034878 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.035363 17882 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:37.035907 17668 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:37.038453 18061 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:37.042799 18045 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:37.043331 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.045053 15085 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:37.058430 15085 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:37.045199 17603 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:37.059720 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:37.047722 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 97544.5us
W20260709 13:03:37.052635 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 100854us
I20260709 13:03:37.053081 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.096755 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2327) took 1217 ms. Trace:
I20260709 13:03:37.097242 15084 rpcz_store.cc:276] 0709 13:03:35.879444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.031475 (+152031us) service_pool.cc:224] Handling call
0709 13:03:37.096496 (+1065021us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:37.096687 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633622272}
I20260709 13:03:37.098389 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.099570 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.068506 15085 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.102129 15085 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.068468 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.059659 18045 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:37.054171 17631 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.087035 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:37.106323 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:37.084558 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.103209 15161 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:37.068982 17882 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:37.120235 17882 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.124480 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.065557 17668 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.124913 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:37.126957 17668 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.127748 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:37.128016 17824 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:37.128621 18087 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.404s	user 0.000s	sys 0.005s
W20260709 13:03:37.132966 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.134744 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:37.136155 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 69799us
I20260709 13:03:37.137243 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:37.138362 18009 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.138943 18009 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:37.142719 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 99508.3us
I20260709 13:03:37.111275 17598 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.118172 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48182.5us
I20260709 13:03:37.118908 17990 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:37.134917 17986 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.161715 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.164256 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.165143 17598 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.166041 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:37.167510 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.167680 17598 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.168236 15847 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:37.168087 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:37.169103 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.169162 15843 raft_consensus.cc:2376] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6. Re-sending same reply.
I20260709 13:03:37.170763 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.171900 15841 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:37.173262 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:37.174157 15848 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.175462 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 773) took 1358 ms. Trace:
I20260709 13:03:37.175751 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 768) took 1520 ms. Trace:
I20260709 13:03:37.175896 15080 rpcz_store.cc:276] 0709 13:03:35.817014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.426393 (+609379us) service_pool.cc:224] Handling call
0709 13:03:36.556723 (+130330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.845201 (+288478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.175186 (+329985us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400192154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:37.175397 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269189376}
I20260709 13:03:37.176092 15082 rpcz_store.cc:276] 0709 13:03:35.655714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.656693 (+   979us) service_pool.cc:224] Handling call
0709 13:03:35.657061 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.764000 (+106939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.175362 (+1411362us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540014e454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:37.175593 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244567808}
I20260709 13:03:37.177151 17867 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:37.177384 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.178936 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.179210 17867 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:37.179891 17867 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:37.180006 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.181797 15083 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:37.181491 14982 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:37.177395 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.182873 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:37.182556 17631 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.184616 18009 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:37.185329 18009 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.186010 18009 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.185534 17867 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.187283 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.188225 15086 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:37.189280 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.190469 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.192233 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.193786 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1706) took 1485 ms. Trace:
I20260709 13:03:37.115856 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.100670 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.143226 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.149408 15079 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:37.152132 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.193838 15237 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:37.195830 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.170800 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:37.202273 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.202178 14982 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 3, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:37.203013 15155 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:37.205626 15154 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:37.194190 15087 rpcz_store.cc:276] 0709 13:03:35.708756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.770702 (+ 61946us) service_pool.cc:224] Handling call
0709 13:03:36.934596 (+1163894us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:36.960266 (+ 25670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.193591 (+233325us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400191c50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:37.193731 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523281664}
W20260709 13:03:37.207520 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 252706us
I20260709 13:03:37.213503 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:37.215288 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.215948 15233 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:37.215140 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:37.217331 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46714.8us
W20260709 13:03:37.219133 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82020.7us
I20260709 13:03:37.152459 17990 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:37.220599 15234 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:37.220824 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:37.221829 15082 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:37.151698 17989 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.205292 18016 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:37.223343 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:37.226222 15161 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.227290 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:37.217469 15236 raft_consensus.cc:2376] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6. Re-sending same reply.
I20260709 13:03:37.233886 17598 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:37.234282 15233 pending_rounds.cc:85] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 2
I20260709 13:03:37.224565 17900 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:37.235071 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 71014.5us
I20260709 13:03:37.234652 15233 pending_rounds.cc:107] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:03:37.235679 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:37.235986 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.235991 15087 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:03:37.234663 15088 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:37.236203 15233 raft_consensus.cc:2889] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:37.237457 14984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 366) took 19247 ms (client timeout 15000 ms). Trace:
I20260709 13:03:37.237627 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.238404 17893 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:37.239612 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2328) took 1359 ms. Trace:
I20260709 13:03:37.240154 15087 rpcz_store.cc:276] 0709 13:03:35.879672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.213397 (+1333725us) service_pool.cc:224] Handling call
0709 13:03:37.235555 (+ 22158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.239259 (+  3704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.239546 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.241010 15157 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:37.237553 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2331) took 1357 ms. Trace:
I20260709 13:03:37.241984 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.245136 17986 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.247618 17986 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:37.248768 17733 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.249337 17733 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:37.250018 18045 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.251345 18045 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:37.255092 17603 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:37.255972 18056 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.234622 15236 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:37.241767 15088 rpcz_store.cc:276] 0709 13:03:35.880358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.214468 (+1334110us) service_pool.cc:224] Handling call
0709 13:03:37.234441 (+ 19973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.237261 (+  2820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.237476 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.260025 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.215175 17896 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:37.264510 18052 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:37.265235 18052 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.265646 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.266057 18078 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:37.234609 17598 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:37.267509 18078 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:37.267963 18009 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:37.268316 17598 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.269045 18009 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.269802 18009 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:37.270148 18009 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:37.271117 18009 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.272475 18009 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.273082 17896 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.273953 18061 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:37.276597 15081 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:37.278160 15848 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:37.234743 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.224658 17990 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:37.280709 15157 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:03:37.282707 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:37.283546 15154 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:37.284464 18101 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:37.257673 15084 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:37.293125 15086 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:37.294903 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:37.281237 15157 pending_rounds.cc:85] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 2
I20260709 13:03:37.296190 15157 pending_rounds.cc:107] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:03:37.296447 15157 raft_consensus.cc:2889] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.281076 15085 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:37.292294 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.231448 18016 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:37.301091 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.301755 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.305100 17900 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:37.305473 15088 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:37.305953 17900 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:37.306686 17900 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:37.307179 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:37.308555 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.297161 15237 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:03:37.293080 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:37.285944 15128 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.623s (SENT) [suppressed 36 similar messages]
I20260709 13:03:37.251579 17733 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:37.312297 15841 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:37.316197 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:37.316553 15228 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:37.316435 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.262003 14982 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 4, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.293876 18101 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:37.300436 18061 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:37.326934 15156 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:37.334779 15847 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:37.330636 15083 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:37.335556 15083 pending_rounds.cc:85] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:03:37.315892 18103 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:37.333375 18106 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:37.346437 18103 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:37.346662 18010 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.347606 18103 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.349618 18106 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:37.349757 18016 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.350970 18016 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:37.353283 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:37.354696 15843 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:03:37.355113 18096 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
W20260709 13:03:37.355890 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 80215.9us
I20260709 13:03:37.356968 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.358430 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:37.359709 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:37.261837 17989 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:37.334115 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96644us
W20260709 13:03:37.339336 18095 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.548s	user 0.000s	sys 0.003s
I20260709 13:03:37.345327 18061 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:37.345976 15083 pending_rounds.cc:107] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:37.353382 17631 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.331014 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.337639 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.375984 15083 raft_consensus.cc:2889] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.387099 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.320014 18078 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 13:03:37.237960 14984 rpcz_store.cc:269] 0709 13:03:17.990440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.990618 (+   178us) service_pool.cc:224] Handling call
0709 13:03:37.237152 (+19246534us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:26.393418 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:26.394643 (+  1225us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:26.396962 (+  2319us) write_op.cc:435] Acquired schema lock
0709 13:03:26.413574 (+ 16612us) tablet.cc:662] Decoding operations
0709 13:03:26.817011 (+403437us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:26.826859 (+  9848us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:26.834010 (+  7151us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:03:26.862335 (+ 28325us) tablet.cc:701] Row locks acquired
0709 13:03:26.862523 (+   188us) write_op.cc:260] PREPARE: finished
0709 13:03:26.862724 (+   201us) write_op.cc:270] Start()
0709 13:03:26.862879 (+   155us) write_op.cc:276] Timestamp: P: 1783602206862660 usec, L: 0
0709 13:03:26.862899 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:03:26.864152 (+  1253us) log.cc:844] Serialized 6375 byte log entry
0709 13:03:27.453151 (+588999us) op_driver.cc:464] REPLICATION: finished
0709 13:03:27.982161 (+529010us) write_op.cc:301] APPLY: starting
0709 13:03:27.982260 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:27.982575 (+   315us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:27.982696 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:27.984510 (+  1814us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:27.991579 (+  7069us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:03:28.004494 (+ 12915us) write_op.cc:312] APPLY: finished
0709 13:03:28.050469 (+ 45975us) log.cc:844] Serialized 113 byte log entry
0709 13:03:28.050682 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.050827 (+   145us) write_op.cc:454] Released schema lock
0709 13:03:28.050892 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:28.051466 (+   574us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.051497 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5095,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":506689,"apply.run_cpu_time_us":4373,"apply.run_wall_time_us":69275,"num_ops":16,"prepare.queue_time_us":2156466,"prepare.run_cpu_time_us":91451,"prepare.run_wall_time_us":693320,"raft.queue_time_us":333740,"raft.run_cpu_time_us":28203,"raft.run_wall_time_us":376150,"replication_time_us":566188,"thread_start_us":2783215,"threads_started":4,"wal-append.queue_time_us":184779,"wal-append.run_cpu_time_us":30402,"wal-append.run_wall_time_us":2009965}]]}
I20260709 13:03:37.391391 15128 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.378525 14982 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 5, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 5 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.387979 15228 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:37.394680 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:37.391631 17787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.012s	sys 0.033s
W20260709 13:03:37.402293 17787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.012s	sys 0.033s
W20260709 13:03:37.402770 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.392144 15234 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:03:37.404379 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51193.7us
I20260709 13:03:37.405143 15229 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:37.393424 17900 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:37.380378 18061 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:37.411022 17824 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:37.413306 17824 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:37.413779 17824 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.414441 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.414891 18052 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.417727 17787 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:37.393908 18103 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:37.485401 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.487000 18106 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:37.418332 18010 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.344487 17880 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.492926 17900 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.434247 17787 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:37.500952 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 61193.9us
I20260709 13:03:37.497309 14982 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 4, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:37.510525 17891 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:37.510694 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.517148 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:37.445274 17823 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:37.523103 15235 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:37.524453 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 164809us
I20260709 13:03:37.525195 15234 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:37.526511 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:37.546002 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54305.7us
I20260709 13:03:37.550776 18106 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:37.551939 17893 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:37.552685 17893 consensus_queue.cc:1243] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:37.529176 17891 consensus_queue.cc:1243] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:37.519538 17787 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.555477 18101 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.557494 17896 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.557566 18101 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:37.547055 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:37.548010 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.560794 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.546384 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.558770 14982 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 4, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.565002 15231 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:37.521082 17986 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:37.565049 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 80016.3us
I20260709 13:03:37.571549 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.569980 17986 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.577441 17986 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:37.580960 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:37.582142 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:37.584026 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.505867 17900 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.522778 18010 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:03:37.569665 17880 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:37.631955 15161 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:03:37.634501 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.652760 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.650849 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.646366 18106 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:37.665221 17823 raft_consensus.cc:2764] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 13:03:37.672400 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.665858 17824 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.630313 17995 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:37.677731 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 446374us
I20260709 13:03:37.675555 17824 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:37.678874 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105250us
I20260709 13:03:37.687759 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:37.694301 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48008.1us
I20260709 13:03:37.692417 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.680375 17995 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:37.699692 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 383296us
I20260709 13:03:37.671245 17899 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 13:03:37.709484 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 293760us
W20260709 13:03:37.713999 18110 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:37.699672 17824 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:37.696614 17989 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:37.718056 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 150829us
I20260709 13:03:37.728466 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:37.733775 17844 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:37.734403 17844 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.734903 17844 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:37.739670 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:37.743417 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:37.745913 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.021s
W20260709 13:03:37.747156 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.011s	sys 0.021s
W20260709 13:03:37.759281 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 65355.6us
I20260709 13:03:37.768690 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:37.770697 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.758733 17882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.013s	sys 0.047s
I20260709 13:03:37.771458 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:37.775650 17867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.009s	sys 0.028s
I20260709 13:03:37.725188 18108 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:03:37.757985 17896 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:03:37.782603 18103 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:37.783215 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:37.784248 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44712.2us
I20260709 13:03:37.790593 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:37.739225 17891 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:03:37.758253 18109 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:37.802906 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:37.799891 17986 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:37.806941 15160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.012s	sys 0.017s
W20260709 13:03:37.807296 15160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.012s	sys 0.017s
I20260709 13:03:37.775059 17598 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:37.777262 17882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.013s	sys 0.047s
I20260709 13:03:37.807109 17896 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:37.787001 17867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.009s	sys 0.028s
I20260709 13:03:37.766652 17995 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:37.842823 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52308.2us
I20260709 13:03:37.800179 17733 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:37.856207 15080 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.013s
W20260709 13:03:37.877113 15080 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.010s	sys 0.013s
W20260709 13:03:37.856941 15162 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.061s	sys 0.076s
I20260709 13:03:37.801414 18103 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:37.878312 15162 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.061s	sys 0.077s
I20260709 13:03:37.783950 17603 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:03:37.880713 15087 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:37.784677 18106 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:37.832015 17989 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:37.892616 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58260.2us
I20260709 13:03:37.833213 17896 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:37.894784 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 111687us
I20260709 13:03:37.877671 17900 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.886341 18103 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:37.834947 17787 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.930641 17787 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:37.906155 18121 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:37.935525 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:37.911845 18009 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.006s
I20260709 13:03:37.913149 17900 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:37.945287 17900 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:37.947386 17900 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:37.906889 17882 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:03:37.948413 17598 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.897097 17989 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.934451 18121 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.895567 17867 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:37.949796 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48939us
I20260709 13:03:37.949494 17896 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:37.903295 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.944052 17844 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:37.955844 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.011s	sys 0.039s
W20260709 13:03:37.971000 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.012s	sys 0.039s
W20260709 13:03:37.968726 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 432442us
I20260709 13:03:37.954298 18078 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:37.953974 17989 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:37.975389 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 289656us
W20260709 13:03:37.976878 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 639437us
I20260709 13:03:37.966670 17603 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:37.975641 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1714) took 1327 ms. Trace:
I20260709 13:03:37.963639 18121 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:37.996778 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 264353us
I20260709 13:03:37.973536 18009 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:03:38.013216 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 300906us
W20260709 13:03:37.998147 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.015s	sys 0.016s
W20260709 13:03:38.021313 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.015s	sys 0.016s
W20260709 13:03:38.013984 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 210220us
I20260709 13:03:38.017745 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:37.975059 17986 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:37.973942 17598 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:38.062731 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 182989us
I20260709 13:03:37.978106 18078 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.065837 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:38.073546 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:38.003080 17603 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:38.076900 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:38.001543 18023 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.075965 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47996.7us
I20260709 13:03:38.090181 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:38.091200 15228 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:38.089397 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 88828.2us
W20260709 13:03:38.074493 18101 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:03:38.097885 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.065405 18009 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:38.086272 18078 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:38.093149 17891 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:38.088431 17896 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:38.084698 18128 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:38.116510 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:38.104723 18045 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.853s	user 0.014s	sys 0.001s
I20260709 13:03:38.099408 17733 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:38.093585 17824 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:38.113457 17891 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:38.123072 18061 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.002s
I20260709 13:03:38.131593 18128 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:38.139534 17824 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.140065 18128 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.140580 17844 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:38.140828 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:38.141827 18125 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:03:38.142623 18125 consensus_queue.cc:1243] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:38.143127 18125 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:38.143242 17598 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:38.143673 18125 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.143899 17598 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:38.145542 18115 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:38.123795 17733 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.147768 18061 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.148146 18009 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.148442 18127 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:03:38.148964 18141 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:38.150776 17823 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:38.151962 17598 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:38.153050 18118 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:03:38.153469 17867 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:38.192286 17867 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:38.192830 15841 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:38.143918 18116 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:03:38.003247 15082 rpcz_store.cc:276] 0709 13:03:36.648635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.229433 (+580798us) service_pool.cc:224] Handling call
0709 13:03:37.245175 (+ 15742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.245536 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:37.740933 (+495397us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.975263 (+234330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.975331 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.975380 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.975445 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.975588 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":725688,"raft.queue_time_us":3753,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":3045,"thread_start_us":685363,"threads_started":2}
I20260709 13:03:38.193310 17867 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.194492 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.194629 17867 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:38.156548 18131 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:38.197167 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.197383 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.197953 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.197290 15844 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:38.158654 18104 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:38.161078 17798 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:38.135810 17603 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.105175 18023 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:38.134649 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.212522 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.168272 17989 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:38.169534 17990 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.855s	user 0.006s	sys 0.007s
I20260709 13:03:38.165758 18045 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.166033 17733 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:38.138779 17891 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:38.191589 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.178107 18135 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:38.202390 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.204514 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:38.261264 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.205919 18061 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.197453 15839 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:38.270577 15086 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:38.221432 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.190593 18141 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.272358 15841 raft_consensus.cc:2410] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:38.203015 17880 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
W20260709 13:03:38.237835 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 336548us
I20260709 13:03:38.240777 18137 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.245455 17914 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.000s	sys 0.010s
I20260709 13:03:38.248317 15848 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:38.254580 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:38.257310 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 239802us
I20260709 13:03:38.201280 18104 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:38.203312 17798 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.256095 17844 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.266171 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.266912 18125 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:38.261273 18045 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:38.273175 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 215652us
I20260709 13:03:38.273818 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1908) took 1149 ms. Trace:
I20260709 13:03:38.525135 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:38.403688 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.525098 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:38.526623 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.510121 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.371541 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 308685us
W20260709 13:03:38.285770 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 107969us
I20260709 13:03:38.306672 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.544688 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:38.546943 15846 raft_consensus.cc:2410] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:38.509248 18103 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.005s
I20260709 13:03:38.247455 18109 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:38.551411 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:38.249799 17989 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:38.534698 17914 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.000s	sys 0.032s
I20260709 13:03:38.369920 17603 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:38.557787 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 303279us
I20260709 13:03:38.531386 17798 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:38.561703 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.558275 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.402269 17831 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:38.525554 15162 rpcz_store.cc:276] 0709 13:03:37.124379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.125921 (+  1542us) service_pool.cc:224] Handling call
0709 13:03:37.126207 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.126550 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:37.886859 (+760309us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.258780 (+371921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.260173 (+  1393us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:38.273281 (+ 13108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.273488 (+   207us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.273698 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23969,"prepare.queue_time_us":750203,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2782,"raft.queue_time_us":91798,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":764288,"thread_start_us":1083548,"threads_started":3,"wal-append.queue_time_us":506299}
I20260709 13:03:38.277189 17631 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:38.568537 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 43515.6us
I20260709 13:03:38.533160 18023 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.403934 18065 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.579191 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.579480 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.579492 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.548172 17896 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.257014 17990 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.593022 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.592604 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:38.557240 18106 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:38.600458 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:38.600895 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:38.493119 17733 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.600488 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:38.607961 15159 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:38.608166 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.616959 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.306878 18078 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.561692 18141 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:38.573275 17831 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:38.626596 15158 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.4-4.5]
I20260709 13:03:38.576303 18132 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:38.640201 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.285725 18135 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.562466 18131 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:03:38.542598 17844 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.559947 18125 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:38.645649 15085 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:38.600148 18158 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:38.647341 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.605703 17896 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.567756 17880 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:03:38.564456 18103 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.658352 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:38.648097 18023 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.606819 18106 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:38.662123 15082 raft_consensus.cc:2368] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:38.538543 18137 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.746639 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:38.747319 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79488.9us
I20260709 13:03:38.675323 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.678617 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:38.727504 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:38.601733 18065 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.647918 18168 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.737874 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.739238 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.749275 15158 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:38.749325 15155 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:38.752306 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:38.778558 15847 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:38.780339 15160 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:03:38.770713 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 98392.4us
I20260709 13:03:38.746058 18106 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:38.771718 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:38.776244 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.726543 18158 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.769963 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:38.727756 18141 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.664994 17823 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:03:38.802544 15841 raft_consensus.cc:2410] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:38.805414 15156 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:38.805673 15163 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:38.805761 18158 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:38.806111 18153 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.806541 18150 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.206s
I20260709 13:03:38.807116 18161 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.807487 18127 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 13:03:38.808350 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 150110us
I20260709 13:03:38.810127 18137 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.813112 15155 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:38.742125 18128 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:38.819690 18156 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:38.820272 18151 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:03:38.822712 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:38.825910 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.772548 18023 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.749140 18132 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:38.847362 15158 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:38.871937 17900 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:38.871297 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 209257us
I20260709 13:03:38.871249 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:38.963595 15160 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.909588 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:38.645519 18104 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.911543 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:38.966657 15155 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.967015 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:38.968976 18165 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.969637 18165 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.841300 15229 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:38.849313 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.917421 18151 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:38.972476 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:38.918678 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.973383 15234 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:38.919494 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.973452 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.921433 17733 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:38.841572 17831 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.924329 17896 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.924827 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:38.926514 18173 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:03:38.929620 15159 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:38.937363 17867 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.742s	user 0.007s	sys 0.005s
I20260709 13:03:38.942339 18065 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.939587 18135 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.941418 17990 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.908280 18161 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:38.803551 18168 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.947386 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.028452 15158 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:38.839643 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:38.947530 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:38.961242 15157 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:38.964054 15847 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:38.972640 18151 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:38.919808 18121 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.846s	user 0.011s	sys 0.001s
I20260709 13:03:38.905437 17598 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.844242 18141 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:38.980429 15163 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:38.987323 15155 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:03:38.963223 17891 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.000s
I20260709 13:03:38.994240 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 777) took 2022 ms. Trace:
I20260709 13:03:38.938396 17995 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:39.001822 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.001822 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:39.006302 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.009008 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 260085us
W20260709 13:03:39.044476 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.017395 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:39.019575 15161 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:03:39.021152 15839 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:38.863919 18016 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.981259 18045 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.915577 18128 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.032015 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.003531 18104 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:39.043481 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:39.052685 15233 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:39.048611 15161 pending_rounds.cc:85] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 3
I20260709 13:03:39.053416 15161 pending_rounds.cc:107] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:03:39.022827 18137 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.027832 17831 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:39.051690 15841 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:39.043569 15087 rpcz_store.cc:276] 0709 13:03:36.971637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.241052 (+269415us) service_pool.cc:224] Handling call
0709 13:03:37.241370 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.889335 (+647965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.993766 (+1104431us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54001b0dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:38.994128 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469872896}
I20260709 13:03:39.053752 15161 raft_consensus.cc:2889] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.054284 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.064301 15158 raft_consensus.cc:2368] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:39.054504 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.021651 18165 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:38.876474 18153 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.060037 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.029856 18065 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:39.069669 15847 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:39.071925 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:39.072583 15085 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:39.072796 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:39.043996 17882 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.077740 15237 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:39.079998 15159 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:39.069563 15840 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:39.057138 18078 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.083250 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:39.088016 15087 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:39.102766 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:39.084059 17603 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.002s
I20260709 13:03:39.104329 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:39.105357 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 122394us
I20260709 13:03:39.110244 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.063805 17844 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.052007 17598 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.119422 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:39.073561 18150 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.115875 15086 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:39.122220 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.055114 17891 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.122974 15158 raft_consensus.cc:2368] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5.
I20260709 13:03:39.058845 18016 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.099858 18165 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:39.134686 15159 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:39.135319 15844 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.005s
I20260709 13:03:39.049965 18121 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.138381 15844 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:39.073179 18137 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.139438 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.142493 18158 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:39.143635 18175 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:03:39.144395 18179 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:03:39.145342 18177 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:03:39.146131 15237 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:39.146762 18016 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.117511 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.132758 17823 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:39.085008 18132 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.150760 18128 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.151309 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.153662 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.156245 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.087160 17882 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.158633 15083 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:39.156394 15087 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:39.162786 15233 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:39.176946 15163 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:39.178391 18154 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.083853 18065 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.135361 18103 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.212563 15229 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
W20260709 13:03:39.242125 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.118119 18151 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:39.567598 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.184742 15160 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:03:39.297793 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.421572 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.205714 15088 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:39.380482 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.426362 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.567251 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.566737 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.565467 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2343) took 1014 ms. Trace:
I20260709 13:03:39.443805 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.619544 15086 rpcz_store.cc:276] 0709 13:03:38.551433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.557891 (+  6458us) service_pool.cc:224] Handling call
0709 13:03:39.565263 (+1007372us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:39.565411 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180126464}
I20260709 13:03:39.120106 17990 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:39.620240 15847 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:39.557004 17787 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:39.626151 17787 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:39.627648 17787 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.629213 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.629388 18153 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:39.630210 17882 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.626448 17891 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.190828 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.626372 15083 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:39.132254 18150 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.565805 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.628338 17598 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.634522 18150 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.619422 15158 raft_consensus.cc:2376] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8. Re-sending same reply.
I20260709 13:03:39.635327 15844 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:39.631067 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.636826 15848 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:39.637013 17823 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:39.637686 15160 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:39.637917 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.638127 17823 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.638577 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.639611 15160 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 3: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:39.635589 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:39.615756 18131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.009s	sys 0.050s
W20260709 13:03:39.641163 18131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.010s	sys 0.051s
I20260709 13:03:39.566218 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1739) took 1301 ms. Trace:
I20260709 13:03:39.641301 18141 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.641654 15082 rpcz_store.cc:276] 0709 13:03:38.264888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.265090 (+   202us) service_pool.cc:224] Handling call
0709 13:03:39.566042 (+1300952us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:39.566160 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269613952}
I20260709 13:03:39.641342 15158 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:39.644044 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.642769 15841 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:39.644965 15158 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:39.645531 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.646301 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.647946 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.640628 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:39.564530 18162 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.633s	user 0.002s	sys 0.012s
I20260709 13:03:39.649158 15160 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:39.642462 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.653425 18161 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:39.653620 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:39.654443 15159 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:03:39.645694 18106 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.837s	user 0.009s	sys 0.001s
I20260709 13:03:39.659397 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:39.665951 15163 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:39.577771 15126 connection.cc:256] RPC call timeout handler was delayed by -1.0885181959999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 13:03:39.659694 15160 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
W20260709 13:03:39.618330 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.662089 15847 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:39.189680 18180 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:03:39.674393 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.677683 15844 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:39.402181 18163 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.437s	user 0.000s	sys 0.000s
W20260709 13:03:39.190408 18176 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:03:39.518826 18009 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:03:39.681224 15232 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.034s	sys 0.078s
I20260709 13:03:39.641335 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.699419 15160 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:39.682899 15814 leader_election.cc:400] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:39.403035 18023 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.624295 17990 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:39.695688 15232 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.034s	sys 0.078s
I20260709 13:03:39.696478 15156 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:03:39.319108 18168 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:39.561260 18101 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:39.677546 18131 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:39.669632 18153 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:39.685109 17882 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.712435 15841 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:39.715771 17882 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.695602 18009 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:39.716981 15841 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:39.717630 18009 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.718758 15841 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:39.719466 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57139.1us
I20260709 13:03:39.720230 15841 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:39.700891 18151 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:39.721501 15841 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:39.681110 17891 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:39.722616 15841 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:39.722579 17891 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.724056 15841 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:39.724330 18191 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:03:39.725147 15841 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:39.725482 18156 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.000s
I20260709 13:03:39.725939 18156 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:39.708894 18131 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:39.726945 15154 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:39.727068 18131 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.728168 18131 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:39.718837 17882 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.705402 17631 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.720227 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:39.734347 15160 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:39.739454 15841 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:03:39.201709 18165 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.743609 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1752) took 1719 ms. Trace:
I20260709 13:03:39.745735 15156 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:39.748420 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:39.674470 18106 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.706578 17990 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:39.742415 15841 pending_rounds.cc:85] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 4
I20260709 13:03:39.707020 18101 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:39.765211 15841 pending_rounds.cc:107] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:03:39.765663 15841 raft_consensus.cc:2917] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:03:39.765862 18121 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:39.766108 15841 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:39.742014 18104 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.683s	user 0.005s	sys 0.004s
I20260709 13:03:39.767132 15848 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:39.768337 15844 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:39.768885 15844 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:39.711833 15163 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:39.744151 15230 rpcz_store.cc:276] 0709 13:03:38.011709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.012370 (+   661us) service_pool.cc:224] Handling call
0709 13:03:38.065585 (+ 53215us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:38.537209 (+471624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.537238 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:38.538087 (+   849us) log.cc:844] Serialized 52 byte log entry
0709 13:03:38.744677 (+206590us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:38.744733 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.744859 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.730541 (+985682us) raft_consensus.cc:1685] finished
0709 13:03:39.730566 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.730752 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72693,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":607576,"threads_started":2,"wal-append.queue_time_us":352313}
I20260709 13:03:39.668404 18154 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.774966 17891 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.774597 18156 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:39.753669 18023 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.790027 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.750136 18127 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:39.806195 18023 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.811499 18023 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.752787 17631 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.769914 15844 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.798549 15847 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:39.830212 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 861) took 1007 ms. Trace:
I20260709 13:03:39.830478 17631 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.734627 18153 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.812826 15230 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:39.819537 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1921) took 1044 ms. Trace:
I20260709 13:03:39.799207 17891 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.821939 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 820) took 2530 ms (client timeout 1785 ms). Trace:
I20260709 13:03:39.823392 15843 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 13:03:39.824107 18169 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.389s	user 0.011s	sys 0.004s
I20260709 13:03:39.825524 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1920) took 1050 ms. Trace:
W20260709 13:03:39.842741 15846 rpcz_store.cc:269] 0709 13:03:37.291702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.291911 (+   209us) service_pool.cc:224] Handling call
0709 13:03:39.821628 (+2529717us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54006ccfd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:39.821816 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859552000}
I20260709 13:03:39.825714 18165 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:39.827140 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.856999 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:39.838887 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:39.791226 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58200us
I20260709 13:03:39.830559 15847 rpcz_store.cc:276] 0709 13:03:38.822453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.671582 (+849129us) service_pool.cc:224] Handling call
0709 13:03:39.671957 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.829486 (+157529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.830006 (+   520us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400771d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:39.830156 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277852032}
I20260709 13:03:39.841225 15154 rpcz_store.cc:276] 0709 13:03:38.775159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.787384 (+ 12225us) service_pool.cc:224] Handling call
0709 13:03:38.787803 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.727296 (+939493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.819187 (+ 91891us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400378054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:39.819455 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2158994688}
I20260709 13:03:39.859656 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:39.860785 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 858) took 1039 ms. Trace:
I20260709 13:03:39.851068 15843 pending_rounds.cc:85] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 5
I20260709 13:03:39.852989 15157 rpcz_store.cc:276] 0709 13:03:38.774884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.774990 (+   106us) service_pool.cc:224] Handling call
0709 13:03:38.947093 (+172103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.031973 (+ 84880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.825319 (+793346us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54003e0650. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:39.825468 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394982784}
W20260709 13:03:39.852133 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.868611 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.865248 15845 rpcz_store.cc:276] 0709 13:03:38.821265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.827187 (+  5922us) service_pool.cc:224] Handling call
0709 13:03:39.826973 (+999786us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:39.860722 (+ 33749us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31276288}
I20260709 13:03:39.869534 15810 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:39.867417 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.864523 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:39.870330 15841 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:39.869589 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:39.871011 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:39.872129 18065 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:39.872740 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.771979 15163 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:39.873737 15847 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:03:39.888411 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.036s	sys 0.069s
W20260709 13:03:39.889030 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.037s	sys 0.069s
I20260709 13:03:39.874647 15843 pending_rounds.cc:107] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:03:39.902319 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:39.866883 18165 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:39.908574 15230 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:39.900825 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.900671 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:39.852861 17891 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:39.882623 18127 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:39.912279 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:39.912597 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44989.1us
I20260709 13:03:39.872169 18154 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:39.910156 15846 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:39.897775 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 279413us
I20260709 13:03:39.889792 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.924794 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.933668 15128 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:39.935777 15841 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:39.907648 15843 raft_consensus.cc:2889] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.922330 17598 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:39.944535 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:39.911931 18165 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:39.951247 18198 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:39.952258 18198 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:39.953047 18198 raft_consensus.cc:2764] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:03:39.942095 18168 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.954315 18168 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.882400 18101 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:39.961277 18106 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:39.962374 18121 consensus_queue.cc:1175] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 13:03:39.963083 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1917) took 1189 ms. Trace:
I20260709 13:03:39.964988 15161 rpcz_store.cc:276] 0709 13:03:38.774017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.774151 (+   134us) service_pool.cc:224] Handling call
0709 13:03:38.774433 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.063381 (+288948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.962858 (+899477us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54001d8654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:39.963031 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562703744}
I20260709 13:03:39.963727 18165 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:39.945647 17798 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.350s	user 0.011s	sys 0.002s
W20260709 13:03:39.999564 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.013185 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79637.6us
I20260709 13:03:40.014205 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:40.010087 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 97864.2us
W20260709 13:03:40.006742 17995 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.944s	user 0.007s	sys 0.003s
I20260709 13:03:39.988405 17798 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.020905 15231 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:03:40.022290 15231 pending_rounds.cc:85] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 4
I20260709 13:03:40.022509 15231 pending_rounds.cc:107] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:03:40.019726 15128 leader_election.cc:400] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:40.023147 15128 leader_election.cc:403] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:40.018152 17598 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:40.037195 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:40.026018 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 179326us
I20260709 13:03:40.039139 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 581) took 1223 ms. Trace:
I20260709 13:03:40.035351 15228 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:40.034809 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.028020 15231 raft_consensus.cc:2917] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:03:40.034464 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.016302 18168 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:40.034560 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.068377 15231 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:40.039549 15159 rpcz_store.cc:276] 0709 13:03:38.816111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.675164 (+859053us) service_pool.cc:224] Handling call
0709 13:03:39.696104 (+ 20940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.696290 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:39.976775 (+280485us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.038072 (+ 61297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.038131 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.038181 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.038903 (+   722us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.039042 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":342861,"thread_start_us":298495,"threads_started":2}
W20260709 13:03:40.070573 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.038133 15086 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:40.042883 15085 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:40.071203 15052 connection.cc:256] RPC call timeout handler was delayed by -1.0265905449999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 13:03:40.074487 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.078006 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.025951 18198 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.082862 15158 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:40.018977 17867 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:03:40.088497 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:40.082540 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 614) took 1161 ms. Trace:
I20260709 13:03:40.035661 18153 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.041059 17631 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:40.040024 17882 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:40.083859 18203 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:03:40.089511 15232 rpcz_store.cc:276] 0709 13:03:38.920664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.704764 (+784100us) service_pool.cc:224] Handling call
0709 13:03:39.705080 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.705270 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:39.987646 (+282376us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.081905 (+ 94259us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.082189 (+   284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.082235 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.082282 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.082351 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.082471 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10989,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":1813,"spinlock_wait_cycles":13612032,"thread_start_us":371146,"threads_started":2}
I20260709 13:03:40.100795 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2466) took 1120 ms. Trace:
I20260709 13:03:40.082659 18188 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:40.092734 18202 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:03:40.089164 18151 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:40.126228 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.088174 17896 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.112538 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.118080 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.134333 15087 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:40.126523 15160 rpcz_store.cc:276] 0709 13:03:38.980925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.742686 (+761761us) service_pool.cc:224] Handling call
0709 13:03:39.743085 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.743354 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:39.977012 (+233658us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.100465 (+123453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.100513 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:40.100559 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.100626 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.100737 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":320237,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"thread_start_us":325198,"threads_started":3}
I20260709 13:03:40.098105 17899 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.149662 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.131299 18188 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:40.104610 18156 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:40.157763 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 69349.4us
I20260709 13:03:40.158746 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:40.159678 15232 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:40.159811 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.133561 17598 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:40.130857 17891 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:40.167548 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.160612 18213 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:03:40.102818 17882 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:40.187100 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.190642 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.144332 18151 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.171988 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.145066 17896 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.173285 18104 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.199304 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 827) took 1251 ms. Trace:
I20260709 13:03:40.199862 15839 rpcz_store.cc:276] 0709 13:03:38.948085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.714626 (+766541us) service_pool.cc:224] Handling call
0709 13:03:39.791738 (+ 77112us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.810785 (+ 19047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.810824 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.811510 (+   686us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.968904 (+157394us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:39.969011 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.969108 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.195951 (+226843us) raft_consensus.cc:1685] finished
0709 13:03:40.198031 (+  2080us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.199214 (+  1183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1440,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":21274,"thread_start_us":123433,"threads_started":1,"wal-append.queue_time_us":183601}
I20260709 13:03:40.202450 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 612) took 1282 ms. Trace:
I20260709 13:03:40.202940 15233 rpcz_store.cc:276] 0709 13:03:38.920090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.679701 (+759611us) service_pool.cc:224] Handling call
0709 13:03:39.697740 (+ 18039us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.747232 (+ 49492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.747282 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.757521 (+ 10239us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.934943 (+177422us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:39.934999 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.935093 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.202200 (+267107us) raft_consensus.cc:1685] finished
0709 13:03:40.202220 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.202384 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6726,"raft.queue_time_us":29081,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":28781,"spinlock_wait_cycles":16128,"thread_start_us":142626,"threads_started":1,"wal-append.queue_time_us":221583}
I20260709 13:03:40.190343 17867 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:40.209721 15234 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:40.208139 15230 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:40.212168 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1754) took 1452 ms. Trace:
W20260709 13:03:40.216380 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.157465 17899 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.212630 15229 rpcz_store.cc:276] 0709 13:03:38.752698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.753033 (+   335us) service_pool.cc:224] Handling call
0709 13:03:40.204933 (+1451900us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b54002779d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:40.205086 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90088704}
I20260709 13:03:40.221411 17882 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:40.209771 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.228490 15087 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.230106 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.222148 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:40.230875 18178 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.312s	user 0.004s	sys 0.007s
W20260709 13:03:40.232432 18210 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:03:40.232600 15233 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:40.233574 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1909) took 1220 ms. Trace:
I20260709 13:03:40.197444 18104 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.193737 17798 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.170534 17891 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:40.220180 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.234113 15233 rpcz_store.cc:276] 0709 13:03:39.012909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.231827 (+1218918us) service_pool.cc:224] Handling call
0709 13:03:40.232314 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.233236 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.233478 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:40.222672 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.235563 15087 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.236032 15229 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:40.257072 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 618) took 1335 ms. Trace:
I20260709 13:03:40.257460 15229 rpcz_store.cc:276] 0709 13:03:38.921798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.235659 (+1313861us) service_pool.cc:224] Handling call
0709 13:03:40.235900 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.256785 (+ 20885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.257019 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.258738 15229 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.259256 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1912) took 1245 ms. Trace:
I20260709 13:03:40.259526 15229 rpcz_store.cc:276] 0709 13:03:39.013592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.258240 (+1244648us) service_pool.cc:224] Handling call
0709 13:03:40.258576 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.259063 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.259212 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.233355 17631 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:40.261061 17631 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:40.261484 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.262234 15087 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.263147 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:40.236784 18211 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:40.210095 17867 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.264552 15087 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:03:40.238744 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.237793 18209 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:40.241660 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 776) took 3269 ms. Trace:
I20260709 13:03:40.243521 17884 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.244024 18056 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:40.265671 15080 rpcz_store.cc:276] 0709 13:03:36.971282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.233602 (+262320us) service_pool.cc:224] Handling call
0709 13:03:37.288832 (+ 55230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.310459 (+ 21627us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:37.879769 (+569310us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.240167 (+2360398us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b1800412a80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a
0709 13:03:40.240341 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.240381 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.240418 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.240511 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.240635 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":699610,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":10427,"spinlock_wait_cycles":6780672,"thread_start_us":721515,"threads_started":2,"wal-append.queue_time_us":311872,"wal-append.run_cpu_time_us":1233,"wal-append.run_wall_time_us":1025347}
I20260709 13:03:40.245316 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 853) took 1808 ms. Trace:
I20260709 13:03:40.245662 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.246364 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.246578 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.247324 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.248056 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.248528 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:40.250465 15232 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:03:40.278115 15233 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:40.264786 17867 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.226732 17899 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:40.218552 17990 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.222235 18218 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:03:40.287858 17990 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.289887 18165 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.290979 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.292429 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.281757 15840 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:40.282956 18151 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:40.293478 15230 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:40.293568 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.294639 15840 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:40.285308 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:40.295488 17896 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.296353 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.297281 15846 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:40.227684 17598 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:40.286795 18131 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.006s
I20260709 13:03:40.306996 18104 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:40.308061 18165 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.279561 15843 rpcz_store.cc:276] 0709 13:03:38.436783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.437812 (+  1029us) service_pool.cc:224] Handling call
0709 13:03:39.822785 (+1384973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.996101 (+173316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.245040 (+248939us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54006ccfd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:40.245244 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3171963264}
I20260709 13:03:40.312394 15232 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:40.310326 15080 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:40.311309 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149153us
I20260709 13:03:40.317036 15232 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:40.318257 15232 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:40.295495 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.320132 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.323088 18198 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:40.324505 17899 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:40.326565 17899 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:40.327075 17899 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:40.328092 17885 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.328799 17885 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.325755 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:40.339917 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 95203.3us
I20260709 13:03:40.342044 15081 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:40.342131 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.347393 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1903) took 1596 ms. Trace:
I20260709 13:03:40.348397 18219 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:40.351827 17882 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:40.352453 17882 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:40.352981 17882 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.358289 17631 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.359584 18137 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.361115 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.361958 15160 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:40.362675 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.363642 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.363958 15154 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:40.364248 17787 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:40.365352 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.366173 15161 raft_consensus.cc:2376] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4. Re-sending same reply.
I20260709 13:03:40.367408 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.368252 15161 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.368909 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.369318 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.370131 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.370460 15157 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:40.362902 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.374819 15156 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:40.377465 17914 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:40.350606 18044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.357s	user 0.019s	sys 0.023s
I20260709 13:03:40.378027 17914 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:40.378831 18044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.456s	user 0.020s	sys 0.024s
I20260709 13:03:40.380369 17914 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:40.381374 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:40.351851 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.364210 15160 raft_consensus.cc:2376] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6. Re-sending same reply.
I20260709 13:03:40.348505 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:40.386320 15201 connection.cc:256] RPC call timeout handler was delayed by -1.3253443820000257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:03:40.387099 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.387456 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.349182 15236 rpcz_store.cc:276] 0709 13:03:38.750791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.750928 (+   137us) service_pool.cc:224] Handling call
0709 13:03:38.842857 (+ 91929us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.107085 (+264228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.107121 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.119880 (+ 12759us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.892098 (+772218us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:39.942419 (+ 50321us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.942646 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.344721 (+402075us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400318654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:40.344774 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.344823 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:40.347121 (+  2298us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.347318 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33962,"prepare.queue_time_us":53668,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":27057,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":14245,"spinlock_wait_cycles":564682496,"thread_start_us":640458,"threads_started":1,"wal-append.queue_time_us":768530}
I20260709 13:03:40.389578 15080 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:40.389791 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.390054 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:40.390654 15203 connection.cc:256] RPC call timeout handler was delayed by -1.2149342809999553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:03:40.391053 15203 connection.cc:256] RPC call timeout handler was delayed by -1.2128404149999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
W20260709 13:03:40.391876 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.392023 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.390488 15154 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.393847 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 894) took 1221 ms. Trace:
I20260709 13:03:40.394225 15842 rpcz_store.cc:276] 0709 13:03:39.165617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.197651 (+ 32034us) service_pool.cc:224] Handling call
0709 13:03:40.351679 (+1154028us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:40.386749 (+ 35070us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35081472}
I20260709 13:03:40.392787 15155 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:40.400548 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.401262 15161 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:40.406883 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.395675 18044 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:40.398041 18165 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.399737 17798 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.426254 18223 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.424402 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:40.465045 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48932us
W20260709 13:03:40.477383 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 128950us
I20260709 13:03:40.478749 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.406126 18131 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:03:40.498118 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 607) took 1582 ms. Trace:
I20260709 13:03:40.498742 15234 rpcz_store.cc:276] 0709 13:03:38.915654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.058363 (+142709us) service_pool.cc:224] Handling call
0709 13:03:40.205190 (+1146827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.210336 (+  5146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.497787 (+287451us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002779d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:40.498025 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636678016}
I20260709 13:03:40.496762 18044 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:40.507071 18044 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.507727 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.507211 17867 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.481185 17631 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.494141 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:40.516644 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100858us
I20260709 13:03:40.518625 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:40.412170 18153 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:03:40.515581 17896 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:40.493505 18156 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:40.510946 15128 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:40.519392 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.511957 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.530728 15230 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:40.516942 18227 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:40.535920 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:40.521771 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:40.410080 15236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "301fa97ba426402092eaf85685303b33"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43954
I20260709 13:03:40.538782 15236 raft_consensus.cc:606] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:03:40.539314 15236 raft_consensus.cc:614] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:40.540480 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.491182 17914 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:03:40.543331 15202 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.991s (SENT)
I20260709 13:03:40.548807 15086 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:40.520752 18184 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:40.497097 17798 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.497012 18165 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.562541 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:40.555675 17631 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:40.566421 17867 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.566584 18223 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:40.559413 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:40.571579 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 616) took 1650 ms. Trace:
I20260709 13:03:40.555485 18219 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:03:40.548399 17598 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:40.546684 17896 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:40.552597 18052 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.578598 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:40.573297 18132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.002s	sys 0.022s
I20260709 13:03:40.593855 18234 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:40.595011 18235 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:40.572407 18227 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:40.602110 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1913) took 1550 ms. Trace:
I20260709 13:03:40.569512 18157 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:03:40.576491 17891 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:40.618225 18165 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:40.612396 18229 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:03:40.604758 18132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.022s
W20260709 13:03:40.613135 18228 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:03:40.613881 18224 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:40.614212 18131 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:40.593115 17598 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.621867 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.622184 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 605) took 1707 ms. Trace:
I20260709 13:03:40.638223 15228 rpcz_store.cc:276] 0709 13:03:38.915006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.915131 (+   125us) service_pool.cc:224] Handling call
0709 13:03:38.974603 (+ 59472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.068662 (+1094059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.621841 (+553179us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400176954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:40.622102 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438917376}
W20260709 13:03:40.640017 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73859.6us
I20260709 13:03:40.611370 15229 rpcz_store.cc:276] 0709 13:03:39.051395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.260152 (+1208757us) service_pool.cc:224] Handling call
0709 13:03:40.260410 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.260604 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:40.273816 (+ 13212us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.427323 (+153507us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b180041c320. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:03:40.600876 (+173553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.601693 (+   817us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:40.601764 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.601853 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.602025 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12982,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":40416,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":6081792,"thread_start_us":123620,"threads_started":2,"wal-append.queue_time_us":279078}
I20260709 13:03:40.597373 17631 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:40.627161 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1902) took 1881 ms. Trace:
W20260709 13:03:40.632886 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.590633 15235 rpcz_store.cc:276] 0709 13:03:38.921216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.725959 (+804743us) service_pool.cc:224] Handling call
0709 13:03:39.726218 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.861771 (+135553us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:39.875723 (+ 13952us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.973084 (+ 97361us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18001dd1c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:03:40.026523 (+ 53439us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.026761 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.026842 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.026913 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.571249 (+544336us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400176e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:40.571325 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.571504 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13200,"prepare.queue_time_us":18723,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":4071,"raft.queue_time_us":333,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":14619,"spinlock_wait_cycles":412931712,"thread_start_us":48189,"threads_started":1,"wal-append.queue_time_us":217908}
I20260709 13:03:40.578768 18153 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:40.643599 15231 rpcz_store.cc:276] 0709 13:03:38.745429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.745638 (+   209us) service_pool.cc:224] Handling call
0709 13:03:38.745972 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.082538 (+1336566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.626814 (+544276us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400295254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:40.627080 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2931151232}
W20260709 13:03:40.647938 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 104655us
I20260709 13:03:40.641557 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:40.608948 18234 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:40.611230 18227 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:40.646376 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.650796 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:40.654398 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.639628 17598 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.670428 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.666777 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.642331 18235 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:40.605764 18052 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.672708 15087 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:40.679343 15086 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.683303 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:40.687033 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:40.687388 15086 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:40.655710 18234 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:40.662299 17914 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:40.688445 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.668388 18065 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.684289 18165 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:40.703402 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.689136 18131 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:40.661854 17631 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.688902 18132 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:03:40.713827 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.714242 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.710765 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:40.712204 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.684281 17884 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:03:40.715927 15080 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:40.703873 18219 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:40.719954 18233 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:40.721382 18235 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.686245 17896 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.711205 18237 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:40.725127 18237 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.727253 18237 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.730775 18052 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:40.732517 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.733141 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:40.734197 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.734319 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:40.734658 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.735140 15083 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:03:40.735523 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.737923 15814 leader_election.cc:400] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5.
I20260709 13:03:40.739760 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.740460 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1911) took 1727 ms. Trace:
I20260709 13:03:40.741381 15233 rpcz_store.cc:276] 0709 13:03:39.013375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.257180 (+1243805us) service_pool.cc:224] Handling call
0709 13:03:40.277930 (+ 20750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.278560 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.740282 (+461722us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54003d9ad0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:40.740405 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38519040}
I20260709 13:03:40.742264 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:40.742552 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.743947 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.744295 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.744625 15161 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:40.745110 15155 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:40.745452 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.746153 15161 raft_consensus.cc:2368] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:40.743633 15086 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:40.705423 18044 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:40.749157 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:40.761091 15156 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:40.761432 15159 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.761965 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.762759 15163 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:40.764168 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:40.716284 18223 raft_consensus.cc:2764] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:03:40.759934 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45758us
I20260709 13:03:40.769821 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:40.759708 15161 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:40.728327 17899 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.748647 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.717803 18219 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.770382 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2347) took 2192 ms. Trace:
I20260709 13:03:40.750140 17787 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:40.749106 18157 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.758858 18121 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:40.778357 17880 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:40.770259 18052 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:40.780755 18139 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.781811 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:40.782305 18139 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.782703 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:40.783671 18132 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:40.784062 18132 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:40.784427 18132 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.779003 15079 rpcz_store.cc:276] 0709 13:03:38.577778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.580876 (+  3098us) service_pool.cc:224] Handling call
0709 13:03:38.581227 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.581470 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.581498 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.582062 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.844076 (+262014us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:38.866878 (+ 22802us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.595787 (+1728909us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b18004193e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:03:40.769266 (+173479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.769340 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.769381 (+    41us) raft_consensus.cc:1685] finished
0709 13:03:40.770166 (+   785us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.770322 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22594,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":5256,"raft.queue_time_us":110764,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":27758,"spinlock_wait_cycles":53859072,"thread_start_us":390226,"threads_started":2,"wal-append.queue_time_us":362058}
I20260709 13:03:40.784440 18139 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.786656 17899 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.775110 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:40.787787 17787 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:40.787976 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.788347 17787 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:40.747987 17882 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.789140 18104 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.789352 17787 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.790740 17995 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:40.790853 17882 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.791276 17995 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.791700 17995 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:40.791494 17880 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:40.749374 18233 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:40.762480 18131 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:40.766202 18153 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:40.808881 15080 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:40.748875 18044 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.811395 18044 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.780721 18121 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.767833 18237 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:03:40.793980 18219 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.828289 18243 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:40.841441 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 153069us
I20260709 13:03:40.825289 18121 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.824803 18131 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:40.839923 18158 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.840291 18243 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:40.821229 18153 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.854115 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:40.855389 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.845055 18215 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:40.860394 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.810978 18157 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.847157 18171 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:03:40.843828 18240 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:03:40.852721 18131 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:40.871268 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:40.874806 15157 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:40.875792 15157 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 6
I20260709 13:03:40.876113 15157 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:40.875886 15084 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:40.876439 15157 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:40.877866 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 95242.7us
I20260709 13:03:40.879091 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:40.869251 15080 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:40.876395 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:40.883209 15233 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:40.886461 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.886724 18245 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:40.889169 18243 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.890046 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.892019 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.892325 15231 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:40.893187 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:40.881281 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:40.893913 18153 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:40.897429 18239 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:03:40.903720 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.876025 18240 consensus_queue.cc:1243] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:40.904417 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:40.908826 15846 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:03:40.860496 18233 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:40.915583 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:40.871390 18215 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:40.902629 18238 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:03:40.898607 18255 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:40.916452 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:40.918241 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:41.097635 15847 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:40.919822 18157 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:40.874104 17598 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:40.898321 15034 auto_rebalancer.cc:893] Removing replace marker failed: Replica 4b633b84351f40de9437e3d532fe6fa7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:03:41.099581 15034 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:03:41.099668 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:40.927125 15234 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:40.931239 15233 raft_consensus.cc:2368] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:41.101696 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.102254 15233 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:40.943064 18245 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:41.049757 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.103045 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:41.103439 15229 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:03:40.931595 18235 raft_consensus.cc:2764] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:03:41.009619 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 216978us
I20260709 13:03:40.924939 17880 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:40.939839 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58659us
I20260709 13:03:41.116133 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:40.924686 18104 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:41.080821 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.117331 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:41.117944 15847 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:41.078215 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 286020us
I20260709 13:03:41.118784 17891 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:41.119177 17891 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:41.119697 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:41.120028 17891 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.082670 15846 pending_rounds.cc:85] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 5
W20260709 13:03:41.049722 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 259455us
I20260709 13:03:41.128338 15846 pending_rounds.cc:107] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:03:41.128648 15846 raft_consensus.cc:2889] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:41.128679 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.130013 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:41.081204 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 205624us
I20260709 13:03:41.082448 18153 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:41.083910 18083 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.109227 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.109908 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.110950 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.111567 17598 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:41.112591 17995 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:41.092164 18128 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.113054 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:41.100057 15034 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:03:41.114099 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.162770 15229 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:41.116899 18101 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.121189 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.123126 18237 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:41.124063 18215 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:41.124169 17884 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:41.124954 18248 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:41.143550 17787 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.132133 18256 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:41.125008 18157 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:41.225579 18157 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:41.228050 18157 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:41.132550 18253 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.133790 17867 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:41.229331 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 108234us
I20260709 13:03:41.229472 18253 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:41.085880 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 181543us
I20260709 13:03:41.138454 15847 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:41.231500 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:41.231657 18253 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.232092 15847 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:41.145565 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.148525 15155 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:41.233127 15233 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:41.233789 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.233201 18245 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.234256 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:41.234876 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.236030 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.094928 18255 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:41.077571 18131 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.161646 15840 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:03:41.169358 18246 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:03:41.170933 18139 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:41.210304 18242 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:41.171380 18251 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:41.171670 18135 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:41.098778 18132 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:41.177892 15843 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:41.178570 15232 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:41.162285 15234 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:41.169387 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:41.184054 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.114915 18234 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:40.565071 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:41.229354 17787 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.187618 18128 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:41.205739 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.126771 17880 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:41.206645 15162 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:41.138978 18153 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:41.211285 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:41.130525 15846 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:41.293995 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:41.213481 18269 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:41.214391 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84366.6us
I20260709 13:03:41.222703 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.295322 15154 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:41.296265 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.297408 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.297811 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.224706 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.299018 15163 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:41.223732 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1724) took 4183 ms. Trace:
I20260709 13:03:41.217877 18237 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:41.149776 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.237241 15087 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:41.162231 18104 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:41.301542 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1916) took 2517 ms. Trace:
I20260709 13:03:41.300287 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.301144 15081 rpcz_store.cc:276] 0709 13:03:37.040514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.293305 (+252791us) service_pool.cc:224] Handling call
0709 13:03:37.293627 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.342587 (+3048960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.223480 (+880893us) spinlock_profiling.cc:243] Waited 3.04 s on lock 0x7b540011a8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:41.223668 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6983661824}
I20260709 13:03:41.305380 15162 rpcz_store.cc:276] 0709 13:03:38.773700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.775651 (+  1951us) service_pool.cc:224] Handling call
0709 13:03:41.190076 (+2414425us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bff8c 00007f0d3c8bfef6 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:41.190249 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.290414 (+100165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.290701 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52283648}
I20260709 13:03:41.238456 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:41.235113 15233 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:41.230651 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.307796 15847 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:41.306284 15156 raft_consensus.cc:2376] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6. Re-sending same reply.
I20260709 13:03:41.311103 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.311895 15127 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:41.311954 15233 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:41.313242 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:41.217687 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66911.3us
W20260709 13:03:41.314528 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 83950.2us
I20260709 13:03:41.315047 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:41.262383 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 122374us
I20260709 13:03:41.245532 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.317099 15161 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:41.171298 17995 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.274914 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.315495 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.318944 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:41.193696 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65096.4us
I20260709 13:03:41.319581 15234 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:41.301403 18153 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.320542 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:41.321666 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:41.322407 15234 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:03:41.324311 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1923) took 2536 ms (client timeout 2816 ms). Trace:
I20260709 13:03:41.225594 18236 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:41.298205 15154 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:03:41.236824 15088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2344) took 2675 ms (client timeout 1991 ms). Trace:
I20260709 13:03:41.281028 18256 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:41.301095 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.230717 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.301375 15846 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:41.325572 15126 leader_election.cc:400] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:41.324941 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:41.340776 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:41.333242 15088 rpcz_store.cc:269] 0709 13:03:38.561245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.561443 (+   198us) service_pool.cc:224] Handling call
0709 13:03:41.236589 (+2675146us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540034a154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:41.236736 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474109312}
I20260709 13:03:41.324680 18268 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.343685 18056 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:41.324738 15158 rpcz_store.cc:269] 0709 13:03:38.788371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.647664 (+859293us) service_pool.cc:224] Handling call
0709 13:03:41.324075 (+1676411us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:41.324226 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997456256}
W20260709 13:03:41.334594 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 103936us
I20260709 13:03:41.345515 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.335983 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.333689 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:41.344216 18056 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:41.367794 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83915.9us
I20260709 13:03:41.391702 15229 raft_consensus.cc:2376] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3. Re-sending same reply.
I20260709 13:03:41.390992 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:41.391206 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:41.331950 18248 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.367700 15156 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:03:41.400617 15847 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
W20260709 13:03:41.401610 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65779us
I20260709 13:03:41.403347 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.325847 17631 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:41.284931 18251 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.417411 18083 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:41.347429 17899 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:41.349140 17995 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.398494 18248 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.351352 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:41.421280 17798 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:41.421943 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:41.472136 15847 raft_consensus.cc:2410] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:03:41.422946 17787 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:41.426576 18044 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.002s
I20260709 13:03:41.345053 17831 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:03:41.427834 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:41.391213 18052 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:41.485106 18052 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.485658 18052 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.428809 18235 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.488549 18056 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.367271 18227 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.490569 18227 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:41.443347 18237 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.493031 18056 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:41.433981 18234 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.402143 18266 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.545497 15847 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:41.434762 15232 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:41.384483 18153 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:41.394872 18121 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 13:03:41.343748 18268 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.337225 18233 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.344282 18267 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:41.445259 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.447369 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:41.332629 18236 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:41.331187 18258 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.295348 18273 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.109s
W20260709 13:03:41.455072 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 134715us
W20260709 13:03:41.457638 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 162389us
I20260709 13:03:41.582018 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:41.331611 17823 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:03:41.583122 18236 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.585052 18236 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.587929 17891 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:41.472177 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68922.1us
I20260709 13:03:41.589887 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:41.440495 18150 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:03:41.428056 18132 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.325855 18255 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.483060 18247 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 13:03:41.461782 18248 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.471258 18157 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.602497 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.613188 15160 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:41.610949 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 277343us
W20260709 13:03:41.612321 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 212541us
I20260709 13:03:41.563413 18233 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.551431 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 205988us
I20260709 13:03:41.396286 17880 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:41.562515 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 115223us
I20260709 13:03:41.630434 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:41.504288 18044 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:41.564910 18271 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:03:41.546690 18215 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:41.565760 18158 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:41.571157 18009 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:41.573312 18128 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:41.573998 18219 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.732s	user 0.006s	sys 0.005s
I20260709 13:03:41.574694 18188 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.590937 15847 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:41.615469 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:41.549357 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.618431 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 252471us
I20260709 13:03:41.695165 15087 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:41.696058 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:41.627162 15158 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:41.554822 18121 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.611281 18104 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:41.634774 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.708243 15236 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:41.554414 18268 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:41.665868 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.601307 18248 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:41.611403 18135 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:41.677109 17867 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.670691 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 88753.5us
I20260709 13:03:41.675081 18101 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:41.717473 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.673092 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:41.675521 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92963.6us
W20260709 13:03:41.718660 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45676.7us
I20260709 13:03:41.719435 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:41.676250 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 61178.2us
I20260709 13:03:41.599413 18267 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:41.723860 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:41.677824 18158 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:41.633034 18233 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:41.619640 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:41.601466 17823 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:41.700908 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:41.635442 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.709897 15163 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:41.710891 15154 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:41.714411 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 99065us
I20260709 13:03:41.757787 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.676188 17880 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:41.670660 18215 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:41.670897 17631 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:41.682502 18157 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:41.685997 18168 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.727511 15814 leader_election.cc:400] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:41.728307 15160 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:41.729146 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:41.775641 15814 leader_election.cc:403] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:03:41.776168 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47118.4us
W20260709 13:03:41.776705 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49372.8us
I20260709 13:03:41.777208 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.735054 18245 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:41.735774 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:41.746155 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126618us
I20260709 13:03:41.760938 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:41.722194 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:41.762169 18158 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.763010 17882 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:41.716992 18135 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:41.782202 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:41.806704 15088 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:41.741429 18273 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.741874 18276 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:41.793567 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:41.772310 18168 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.729331 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:41.703814 18219 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.756690 18227 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:41.806528 15844 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:41.760228 18128 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:41.959980 18121 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:41.765280 15840 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:41.814002 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:41.794538 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58890.6us
I20260709 13:03:41.817189 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:41.817476 15087 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:41.818193 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:42.013258 15232 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.734372 18188 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:41.730597 17867 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:41.821334 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.023471 15086 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:41.950850 18276 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:41.812768 18275 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:03:41.940267 17598 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.940585 18154 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.909251 18243 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:41.943924 18283 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:03:41.990267 17885 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:41.809832 18135 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.806912 17882 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:41.802289 17899 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:03:41.990932 18288 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:03:41.857908 18267 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:42.060971 18289 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
I20260709 13:03:42.061205 17867 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:42.061884 17867 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:42.062952 17867 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:42.063483 17867 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:41.857335 18251 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:41.906270 15230 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.797577 18157 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:42.064973 18291 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:03:42.040120 18243 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.065294 18157 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.066071 18243 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:41.871862 15079 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:41.807478 18237 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:41.790222 18269 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:41.875376 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.046995 17598 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.070140 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.070214 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.070851 15079 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:42.070948 15083 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:42.071203 17598 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.072232 15079 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:42.074153 15083 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:42.075567 18253 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:42.076025 18253 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:41.765515 17823 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.076953 15158 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:42.077697 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:42.078884 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:42.079612 15128 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:42.080994 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:42.082623 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:41.911882 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:41.980432 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 201297us
I20260709 13:03:41.812166 18273 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:41.854915 18279 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:03:41.945401 15160 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:42.093300 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:42.032708 18131 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:41.819550 18286 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:41.943109 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:41.850205 17831 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:03:41.867259 18266 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:41.865756 18104 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:41.723030 18101 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:41.959959 18153 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.016265 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:42.013669 15235 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:42.017226 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 203287us
W20260709 13:03:42.025703 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.097193 18273 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:42.024394 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.092669 18137 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:41.966310 18052 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.008684 18044 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:42.034168 18235 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.003s
I20260709 13:03:42.145038 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 600) took 1400 ms. Trace:
I20260709 13:03:42.076169 15083 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:42.084522 15158 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:42.089107 15844 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:42.159650 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 826) took 3313 ms. Trace:
I20260709 13:03:41.792181 18245 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.164216 18101 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.165066 18245 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:42.166213 18101 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.097718 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:42.080278 18242 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:42.169200 18242 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:42.169415 18235 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:42.170414 18235 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.170987 18235 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:42.173789 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:42.042881 18236 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:42.174634 15159 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:42.086957 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:41.981282 18215 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.055147 17882 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:42.026326 18218 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:03:42.123057 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:42.028561 15163 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:42.089154 18188 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.044831 18154 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.099021 18065 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:42.033525 15232 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:42.125784 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.064297 17867 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.106139 18131 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.135617 15235 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:42.136185 17823 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:42.137418 17631 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:42.138598 18153 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:42.100883 17598 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:42.147682 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 876) took 1018 ms. Trace:
I20260709 13:03:42.147913 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:42.281870 15126 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.018s (SENT)
I20260709 13:03:42.157192 15085 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:41.853442 18168 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:42.159125 15157 rpcz_store.cc:276] 0709 13:03:40.744179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.748909 (+  4730us) service_pool.cc:224] Handling call
0709 13:03:40.749304 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.883603 (+134299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.144714 (+1261111us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:42.144958 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287481728}
I20260709 13:03:42.162617 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:42.174113 18184 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:42.176400 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:42.160118 15844 rpcz_store.cc:276] 0709 13:03:38.846288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.702254 (+855966us) service_pool.cc:224] Handling call
0709 13:03:39.730111 (+ 27857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.806366 (+2076255us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b58004c9dd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9400aa 00007f0d3f9c2447 00007f0d3f9dafe7 00007f0d3f9dc7de 00007f0d3f9e6b49 00007f0d3f9e7416 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4
0709 13:03:42.159444 (+353078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.159601 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313289984}
I20260709 13:03:42.093303 18156 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:03:42.281466 18240 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.230831 15082 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:42.179243 15234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.022s
W20260709 13:03:42.454118 15234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.014s	sys 0.022s
W20260709 13:03:42.178305 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.454802 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.151319 18281 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:42.456146 18281 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.458552 18281 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:42.191614 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 104728us
I20260709 13:03:42.313174 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:42.375190 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.461519 18298 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
I20260709 13:03:42.409523 15159 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:42.463202 15159 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:42.462693 18083 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.310829 15848 rpcz_store.cc:276] 0709 13:03:41.129466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.482738 (+353272us) service_pool.cc:224] Handling call
0709 13:03:41.483154 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:41.483633 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.483672 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:41.484465 (+   793us) log.cc:844] Serialized 52 byte log entry
0709 13:03:41.756260 (+271795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:42.017313 (+261053us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.017534 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.017615 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.147441 (+129826us) raft_consensus.cc:1685] finished
0709 13:03:42.147465 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.147608 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35831,"prepare.queue_time_us":412738,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":26661,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7023,"thread_start_us":488167,"threads_started":2,"wal-append.queue_time_us":652131}
I20260709 13:03:42.464679 15159 pending_rounds.cc:85] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 3
I20260709 13:03:42.467407 15159 pending_rounds.cc:107] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 13:03:42.467702 15159 raft_consensus.cc:2917] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
W20260709 13:03:42.467486 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 185673us
I20260709 13:03:42.468180 15159 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:42.469698 18305 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:42.447160 18267 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.471146 18223 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:42.471655 18223 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:42.471457 18267 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.472597 18223 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.473886 18267 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.473855 18223 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:42.476267 18248 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.000s
I20260709 13:03:42.464039 18083 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.290982 18219 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.310746 17598 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.410538 15086 raft_consensus.cc:2376] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5. Re-sending same reply.
I20260709 13:03:42.289474 17631 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:42.453219 15052 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.601s (SENT) [suppressed 33 similar messages]
W20260709 13:03:42.460629 18297 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.334s	user 0.000s	sys 0.000s
I20260709 13:03:42.408444 15083 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:42.479703 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1925) took 1569 ms. Trace:
I20260709 13:03:42.313535 18246 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.310279 18251 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:42.452128 17867 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.105943 18268 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.308611 18272 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.194236 18009 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 13:03:42.402937 17831 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.495546 15086 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:42.482203 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:42.410019 18184 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:42.499538 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.015s	sys 0.018s
I20260709 13:03:42.501258 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:42.492654 17598 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:42.503180 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.008s	sys 0.026s
I20260709 13:03:42.313030 18188 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.497184 15234 rpcz_store.cc:276] 0709 13:03:40.910016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.323777 (+413761us) service_pool.cc:224] Handling call
0709 13:03:41.324030 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.324221 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:41.616653 (+292432us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.479200 (+862547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.479253 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:42.479301 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.479471 (+   170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.479637 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24556,"prepare.queue_time_us":448685,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":26751,"thread_start_us":1129734,"threads_started":2,"wal-append.queue_time_us":562138}
I20260709 13:03:42.493970 18237 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.489566 18258 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.517066 15846 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:42.558629 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 841) took 1464 ms. Trace:
I20260709 13:03:42.559182 15846 rpcz_store.cc:276] 0709 13:03:41.093997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.335094 (+241097us) service_pool.cc:224] Handling call
0709 13:03:42.501064 (+1165970us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:42.558554 (+ 57490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13280128}
I20260709 13:03:42.453475 18215 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:42.517534 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.015s	sys 0.018s
I20260709 13:03:42.534250 18131 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:42.495354 17631 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:42.537781 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.008s	sys 0.026s
I20260709 13:03:42.539033 15842 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:42.553717 18083 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.554252 18052 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:42.500120 18305 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.556588 15162 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:03:42.476778 18248 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:42.562388 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 785234us
I20260709 13:03:42.505827 18135 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.482293 18240 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.537066 18235 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:42.568436 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:42.546362 18270 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:42.625180 15234 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:42.507573 18246 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.558069 18251 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:42.561758 18158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.012s	sys 0.042s
I20260709 13:03:42.629974 15156 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:42.562757 18265 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.622707 18243 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 13:03:42.630908 15079 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:42.630746 15080 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:03:42.650799 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.659076 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.548722 17831 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.522475 18272 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.562009 18267 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:42.689986 18251 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.671267 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2365) took 1329 ms. Trace:
I20260709 13:03:42.694886 15082 rpcz_store.cc:276] 0709 13:03:41.341569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.341808 (+   239us) service_pool.cc:224] Handling call
0709 13:03:42.670869 (+1329061us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54000dbbd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:42.671153 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922178816}
I20260709 13:03:42.698740 18255 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:42.564102 18258 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.707861 18258 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.623035 18305 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.712355 18305 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:42.713773 18313 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:03:42.717531 18314 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:03:42.735131 18248 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.764863 18240 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.768268 18154 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:42.770264 18246 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:42.777827 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.778916 18168 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.780342 18131 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:42.786145 18268 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.663179 18265 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.678038 18243 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.625612 18245 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:42.793973 18101 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 13:03:42.562942 18237 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.854733 18243 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.856403 15087 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:42.805909 15839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.033s	sys 0.070s
W20260709 13:03:42.633421 18158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.013s	sys 0.042s
I20260709 13:03:42.744442 18135 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:42.814729 17831 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:42.816329 15229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1767) took 1920 ms (client timeout 2018 ms). Trace:
W20260709 13:03:42.837318 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 323081us
W20260709 13:03:42.822481 18233 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.059s	user 0.008s	sys 0.004s
I20260709 13:03:42.856968 15161 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:03:42.856966 15839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.033s	sys 0.070s
W20260709 13:03:42.860394 15229 rpcz_store.cc:269] 0709 13:03:40.895901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.163437 (+267536us) service_pool.cc:224] Handling call
0709 13:03:42.816113 (+1652676us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54002779d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:42.816256 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226192768}
I20260709 13:03:42.862239 15157 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:42.860872 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:42.828490 18240 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.865200 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:42.867412 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:42.827347 18156 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:42.836925 18131 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:42.874550 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:42.854755 18245 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:42.777671 17882 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.868193 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1936) took 1046 ms. Trace:
I20260709 13:03:42.880739 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:42.881433 15231 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:42.879436 15230 rpcz_store.cc:276] 0709 13:03:41.821644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.306008 (+484364us) service_pool.cc:224] Handling call
0709 13:03:42.450246 (+144238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.450526 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:42.493522 (+ 42996us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.867861 (+374339us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.867920 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:42.867965 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.868024 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.868142 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42778,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":2391,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":4526,"thread_start_us":373567,"threads_started":1}
I20260709 13:03:42.869207 18243 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:42.860008 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.005s	sys 0.020s
W20260709 13:03:42.861279 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 408067us
I20260709 13:03:42.874979 18154 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:42.872604 18227 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:42.884541 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:42.900018 15084 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:42.889776 18242 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.720s	user 0.009s	sys 0.002s
I20260709 13:03:42.907011 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:42.886566 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.006s	sys 0.020s
I20260709 13:03:42.910203 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.910916 15080 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:42.874521 18270 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:03:42.908983 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 828587us
I20260709 13:03:42.913046 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1914) took 2513 ms. Trace:
I20260709 13:03:42.876375 18156 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:42.915551 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.917610 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:42.924510 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:42.925442 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:42.906821 15230 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:42.882710 18153 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:03:42.875187 18240 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.869498 18233 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:42.922508 18273 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.765s	user 0.011s	sys 0.001s
W20260709 13:03:42.918921 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.005s	sys 0.051s
W20260709 13:03:42.952683 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.006s	sys 0.051s
I20260709 13:03:42.899253 18154 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:42.907732 18242 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.937769 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:42.930872 18223 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:42.972007 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.973493 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:42.921355 18245 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:03:42.937289 17831 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:42.943612 18248 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:42.948094 15083 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:42.990989 15083 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:42.991484 15083 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:42.928300 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:42.949608 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.909116 18227 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:42.921238 17914 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:42.977465 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:42.928483 18282 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:42.920552 15237 rpcz_store.cc:276] 0709 13:03:40.399333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.399607 (+   274us) service_pool.cc:224] Handling call
0709 13:03:40.412963 (+ 13356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.413177 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:40.413496 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.092922 (+1679426us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18003f08e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:03:42.912617 (+819695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.912680 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:42.912725 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.912827 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.912983 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39666,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":72999,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12348,"spinlock_wait_cycles":8422784,"thread_start_us":809670,"threads_started":2,"wal-append.queue_time_us":2413103}
I20260709 13:03:42.922957 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1926) took 1982 ms. Trace:
I20260709 13:03:42.955633 17867 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.983518 15080 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:42.966639 18237 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:42.958492 18128 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:42.958860 18158 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:42.970873 18154 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:42.996024 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.954936 18320 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:03:42.996138 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:42.944705 18268 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:03:42.919245 18131 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.997704 15083 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.958110 18135 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:03:42.980129 18223 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:43.002691 18245 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.000245 17831 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:43.010011 15052 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.770s (SENT)
I20260709 13:03:42.985154 18266 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:42.963567 18273 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:42.988965 18330 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:42.997743 15233 rpcz_store.cc:276] 0709 13:03:40.940739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.328671 (+387932us) service_pool.cc:224] Handling call
0709 13:03:41.329038 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.329312 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:41.671314 (+342002us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.922618 (+1251304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.922692 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:42.922739 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.922796 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.922898 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26387,"prepare.queue_time_us":483848,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":67,"raft.queue_time_us":99253,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":23666,"thread_start_us":1531072,"threads_started":3,"wal-append.queue_time_us":991797}
I20260709 13:03:43.015983 15083 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:43.028143 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.017935 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.020816 15082 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:03:42.984103 18153 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.030527 15082 pending_rounds.cc:85] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:03:43.030864 15082 pending_rounds.cc:107] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:03:43.031677 15082 raft_consensus.cc:2889] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:43.030821 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.021564 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:43.007445 18168 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.024935 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1756) took 2140 ms. Trace:
W20260709 13:03:43.025074 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.003186 17867 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.037401 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.038471 15233 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:43.040032 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.041472 18156 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.009192 18128 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.043989 18128 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.048033 17867 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.049698 18227 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.050580 18258 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.052307 17914 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:43.052899 17914 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.054600 18273 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.054965 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1942) took 1030 ms. Trace:
I20260709 13:03:43.025952 18223 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.058429 17880 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.058954 17880 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.059581 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.027314 18321 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:43.035529 15228 rpcz_store.cc:276] 0709 13:03:40.884063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.884172 (+   109us) service_pool.cc:224] Handling call
0709 13:03:40.887462 (+  3290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.489144 (+601682us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:41.489178 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.489750 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.685772 (+196022us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:41.685808 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.016694 (+1330886us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b5400176454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:43.016751 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.016802 (+    51us) raft_consensus.cc:1685] finished
0709 13:03:43.016861 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.024862 (+  8001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383016192,"thread_start_us":195637,"threads_started":1,"wal-append.queue_time_us":313182}
I20260709 13:03:43.055650 15160 rpcz_store.cc:276] 0709 13:03:42.024976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.094539 (+ 69563us) service_pool.cc:224] Handling call
0709 13:03:42.094863 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.488782 (+393919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.488828 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.490154 (+  1326us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.869951 (+379797us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:42.982372 (+112421us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.982630 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.982707 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.054640 (+ 71933us) raft_consensus.cc:1685] finished
0709 13:03:43.054672 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.054879 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12365,"prepare.queue_time_us":159652,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":186035,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":125850,"thread_start_us":807164,"threads_started":3,"wal-append.queue_time_us":454412}
I20260709 13:03:43.066232 15231 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:43.066913 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.068379 15160 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:43.070175 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:43.071380 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61407.1us
I20260709 13:03:43.030730 18153 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.073751 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:43.078575 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:43.061370 18319 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:03:43.074810 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.075140 15237 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:43.079980 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 995681us
I20260709 13:03:43.010964 18158 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:43.081450 15203 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.638s (SENT)
I20260709 13:03:43.083622 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.084028 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:43.088078 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.088945 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:43.088935 15237 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.091444 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:43.092185 18323 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:03:43.092756 18281 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:43.093251 18281 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.093725 18281 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:43.096611 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:43.097661 15083 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:43.096973 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1935) took 1280 ms. Trace:
I20260709 13:03:43.099918 15232 rpcz_store.cc:276] 0709 13:03:41.821260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.305990 (+484730us) service_pool.cc:224] Handling call
0709 13:03:42.306451 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.306916 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.306961 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.307549 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.809039 (+501490us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:42.809082 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.809153 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.093246 (+284093us) raft_consensus.cc:1685] finished
0709 13:03:43.093906 (+   660us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.096460 (+  2554us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6743,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":44725,"thread_start_us":343998,"threads_started":1,"wal-append.queue_time_us":522314}
I20260709 13:03:43.101750 15229 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:43.029484 18245 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.027338 18266 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.103190 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.095839 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:43.103819 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.075546 17882 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.097440 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.032516 18330 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.110033 18272 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.109530 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.076467 18269 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:43.112104 15159 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:03:43.117591 18307 raft_consensus.cc:670] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:43.113755 17831 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.109834 18171 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:43.102835 18331 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:43.113404 17867 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:43.145138 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.119431 18150 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.122278 18306 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:43.122979 18265 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.102094 18268 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:43.135931 15083 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:43.187673 15083 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:43.129142 18009 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:43.144626 18325 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:03:43.188469 15083 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.135857 18128 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:43.108161 18301 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:03:43.110394 17880 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.165081 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:43.201229 18156 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:43.201570 15082 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.201845 18156 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.142118 18233 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.142891 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.204059 18156 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.204202 15081 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:43.204587 18320 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:43.205158 18320 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.207239 18320 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.207769 18320 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.210225 15082 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:03:43.192277 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.122982 18223 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.142988 18240 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.200570 15083 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.212009 15159 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:43.196547 17787 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:43.189311 18306 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.225510 18223 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.190428 17867 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:43.188544 18331 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:43.189826 18101 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.228722 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 125214us
W20260709 13:03:43.231323 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 83806.7us
I20260709 13:03:43.195469 18242 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.208221 18320 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.232384 17986 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:03:43.241981 15201 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.636s (SENT)
I20260709 13:03:43.238770 18101 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.245366 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:43.246601 18320 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.246745 18233 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:43.246986 18101 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.247305 18233 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.247871 18153 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.214376 18316 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.251338 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 682608us
W20260709 13:03:43.252956 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.256166 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.257432 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.258037 15844 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:43.237524 15083 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:03:43.258947 15083 pending_rounds.cc:85] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:03:43.259168 15083 pending_rounds.cc:107] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:03:43.259387 15083 raft_consensus.cc:2889] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:43.197014 18009 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.239037 18158 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.192811 18150 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.246623 18270 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:43.262951 18255 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.000s
I20260709 13:03:43.277251 18009 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:43.280193 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1761) took 2388 ms (client timeout 2007 ms). Trace:
W20260709 13:03:43.297230 15236 rpcz_store.cc:269] 0709 13:03:40.891449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.891667 (+   218us) service_pool.cc:224] Handling call
0709 13:03:43.279970 (+2388303us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400175a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:43.280111 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772625536}
I20260709 13:03:43.281605 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1938) took 1227 ms. Trace:
I20260709 13:03:43.298552 15235 rpcz_store.cc:276] 0709 13:03:42.053843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.309020 (+255177us) service_pool.cc:224] Handling call
0709 13:03:42.449765 (+140745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.454598 (+  4833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.454626 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.454665 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:42.454702 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.281233 (+826531us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:43.281294 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.281451 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10594048}
I20260709 13:03:43.251178 18153 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:43.305414 17867 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:43.307431 17631 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.005s
I20260709 13:03:43.307904 17631 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.309015 15154 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:43.311784 18337 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:43.267763 18274 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:03:43.312373 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.315321 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.315887 18121 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:43.316195 15236 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:43.263379 18316 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:43.317565 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 830) took 2581 ms (client timeout 2061 ms). Trace:
I20260709 13:03:43.317754 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:43.318008 15842 rpcz_store.cc:269] 0709 13:03:40.736364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.741139 (+  4775us) service_pool.cc:224] Handling call
0709 13:03:43.317388 (+2576249us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54005fd554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:43.317500 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4075064064}
I20260709 13:03:43.316362 18121 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:43.320454 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.276620 18306 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.318929 18316 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.323467 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 845) took 1694 ms. Trace:
I20260709 13:03:43.233325 18154 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:03:43.327751 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.328403 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.330075 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.330679 15841 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:43.331974 18308 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.332300 18009 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.332860 18009 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.333248 18340 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:43.334208 18306 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:43.334578 18306 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:43.334888 18009 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.335532 18306 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.332535 18308 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.336722 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.337095 18306 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.338269 15082 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:43.338742 18308 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.339452 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.340273 15083 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:43.327262 15839 rpcz_store.cc:276] 0709 13:03:41.628753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.631679 (+  2926us) service_pool.cc:224] Handling call
0709 13:03:41.675400 (+ 43721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.675778 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:42.178748 (+502970us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.187642 (+1008894us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.189805 (+  2163us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.190027 (+   222us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.190079 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.190178 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.323166 (+132988us) raft_consensus.cc:1685] finished
0709 13:03:43.323199 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.323392 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":369431,"prepare.queue_time_us":495232,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":27,"raft.queue_time_us":333327,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":22411,"thread_start_us":1162131,"threads_started":2,"wal-append.queue_time_us":776242}
I20260709 13:03:43.334806 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.344295 15842 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:43.344395 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.345573 15845 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:43.346195 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:43.351471 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 98297us
I20260709 13:03:43.354166 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.354893 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.354934 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.355475 15844 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:43.355871 15082 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:43.355662 15839 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:43.355289 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.358316 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.358856 15839 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:43.379079 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.379770 15844 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.385169 15844 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:43.385818 15845 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:43.386803 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.386933 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.387516 15237 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:43.387606 15844 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:43.387619 15845 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 3.
I20260709 13:03:43.389230 18330 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.389772 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.352273 18331 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.390578 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.391382 15845 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:43.391086 18184 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.392354 18184 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:43.358937 15847 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.388305 17914 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.397567 15839 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:43.397935 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44240.8us
I20260709 13:03:43.398723 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.389904 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.394239 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.320147 18153 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.400576 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.400792 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.378993 17867 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.399561 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.399603 15839 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:43.402302 18245 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.264279 18269 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.410521 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.416350 15845 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:43.417670 15847 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:43.408084 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.414062 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.418852 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.420169 15232 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:43.419071 18308 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:43.400478 18227 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.421865 15236 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:43.421252 18009 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:43.419672 18255 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:03:43.423141 18332 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:43.422952 17986 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:43.423529 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1944) took 1398 ms. Trace:
I20260709 13:03:43.436407 15157 rpcz_store.cc:276] 0709 13:03:42.025611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.407392 (+381781us) service_pool.cc:224] Handling call
0709 13:03:42.407795 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.863962 (+456167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.423253 (+559291us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540037e954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:43.423444 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1044665600}
I20260709 13:03:43.423805 18171 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:03:43.438482 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.420917 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.417536 17867 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:43.427248 18338 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:03:43.428259 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.444307 15845 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:43.406823 18233 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.413269 18274 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:43.406121 18215 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:43.451438 18215 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.401279 18265 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:03:43.452641 17867 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.453583 18265 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:43.446162 18274 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.454082 18265 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:43.454293 18274 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:43.444396 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:43.447813 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.450385 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.455955 15845 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:43.456370 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.450467 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.457713 15844 raft_consensus.cc:2376] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6. Re-sending same reply.
I20260709 13:03:43.427991 18308 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:43.460929 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.461483 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.462116 15157 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.463085 18307 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:43.442669 18150 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.463110 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.406044 18121 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.481943 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:43.471117 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.480409 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:43.480444 15845 raft_consensus.cc:2376] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7. Re-sending same reply.
W20260709 13:03:43.463593 17598 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.905s	user 0.012s	sys 0.000s
I20260709 13:03:43.494606 17598 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.495788 18282 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:43.496315 18282 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.438563 18009 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:43.432724 18227 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.490638 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.491636 18154 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.491834 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.492815 18153 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.486399 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.507726 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:43.568200 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 466743us
I20260709 13:03:43.582551 15154 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:43.463169 17631 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:43.551716 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.479110 18307 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:43.438892 18253 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.571241 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.569495 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.593765 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.463919 17867 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:03:43.638408 15159 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:43.639711 15080 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
W20260709 13:03:43.630681 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.642073 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.642802 15844 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:43.643548 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.644052 15844 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:43.699322 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.582379 18242 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.701678 18242 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.485725 18316 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:03:43.457768 18265 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.780793 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:43.771291 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68604.3us
I20260709 13:03:43.741411 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.733510 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.783917 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:43.774672 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.777607 15157 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:03:43.703217 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:43.757695 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68461.1us
I20260709 13:03:43.779904 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:43.766708 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.784071 15229 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:43.786234 15844 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:43.786238 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.787177 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.780072 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:43.788832 15081 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:43.790367 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:43.790961 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.568651 18158 raft_consensus.cc:2764] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:43.816058 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:43.798398 15082 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:43.817330 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 325563us
I20260709 13:03:43.818310 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.801618 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.609576 18307 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:43.607765 18281 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:43.606767 18246 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.837152 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57287.2us
I20260709 13:03:43.782622 17914 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.782038 18154 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:43.820986 18321 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:43.821947 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:43.824282 15155 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:43.812177 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.826320 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.828616 18233 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:43.828928 18331 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:43.829339 17631 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:43.829883 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.868160 15157 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.830471 18330 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:43.830544 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2496) took 2134 ms (client timeout 1991 ms). Trace:
W20260709 13:03:43.869663 15158 rpcz_store.cc:269] 0709 13:03:41.688436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.698104 (+  9668us) service_pool.cc:224] Handling call
0709 13:03:43.823085 (+2124981us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540013f6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:43.823187 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794809600}
I20260709 13:03:43.871099 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.871793 15158 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:43.822574 18327 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:43.743899 18253 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:43.737339 18264 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:43.646222 17882 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.735450 18227 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.614738 17598 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:43.806349 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.790022 18301 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.907775 18242 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.850171 15233 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:43.834407 18247 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:03:43.788409 18156 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:43.860755 15229 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:43.909202 18156 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.909741 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.910894 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.911512 15229 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:43.912403 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:43.862912 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.875171 18330 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:43.888003 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.913100 18330 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:43.913650 18281 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:43.853880 17831 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:03:43.736145 18343 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.393s
I20260709 13:03:43.839632 18265 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:43.917683 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.918730 18275 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:43.919184 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:43.920483 18275 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:43.921037 18275 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.921514 18275 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:43.921653 18350 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
W20260709 13:03:43.923517 18349 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:03:43.870667 18246 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.894764 18158 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:43.907678 18301 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.820657 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.919355 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.923615 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:43.903555 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:43.870469 18341 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:43.928194 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 122678us
I20260709 13:03:43.929597 15844 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:43.935961 15158 raft_consensus.cc:2376] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4. Re-sending same reply.
I20260709 13:03:43.905723 17882 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.943535 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:43.736481 18308 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.946039 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.947701 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:43.953861 15841 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:43.954069 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.866824 18321 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.950243 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:43.875422 18331 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.875934 18233 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:43.875687 18267 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.948096 15157 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:43.935765 18344 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.492s	user 0.001s	sys 0.001s
I20260709 13:03:43.957247 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:43.991466 15157 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:43.964373 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:44.002488 15229 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:43.991884 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:43.969511 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.944862 18341 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:44.012040 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.995707 18267 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.957489 18331 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.874997 18154 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.014495 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:43.999181 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1774) took 1126 ms. Trace:
I20260709 13:03:44.007575 15154 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
W20260709 13:03:44.009774 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 191548us
W20260709 13:03:44.009814 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52643.1us
I20260709 13:03:43.965829 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:43.998826 15848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.026s	sys 0.046s
W20260709 13:03:44.021222 15848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.026s	sys 0.046s
I20260709 13:03:44.015848 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:44.022832 15157 raft_consensus.cc:2410] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:44.022915 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:44.015403 15237 rpcz_store.cc:276] 0709 13:03:42.872745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.394044 (+521299us) service_pool.cc:224] Handling call
0709 13:03:43.394383 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.400972 (+  6589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.401013 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.401509 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.635497 (+233988us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.635612 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.635713 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.998867 (+363154us) raft_consensus.cc:1685] finished
0709 13:03:43.998893 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.999101 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6176,"raft.queue_time_us":6992,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":229105,"threads_started":1,"wal-append.queue_time_us":590772}
I20260709 13:03:44.025656 15237 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:44.026330 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1940) took 1093 ms. Trace:
I20260709 13:03:44.026669 15237 rpcz_store.cc:276] 0709 13:03:42.933094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.025136 (+1092042us) service_pool.cc:224] Handling call
0709 13:03:44.025454 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.026094 (+   640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.026278 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.895248 18327 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:43.955039 18243 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:44.028187 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.950219 18264 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.934235 18356 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:44.027969 15237 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:44.039165 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:43.990070 18223 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:43.990302 17631 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:44.052874 15157 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:44.054368 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:44.055192 15157 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:44.056944 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:44.057773 15154 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:44.061318 18246 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:43.965735 18233 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:44.024652 17831 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:43.952498 18158 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:43.960103 18242 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:43.957132 18321 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:44.041540 15841 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:44.043316 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1941) took 1109 ms. Trace:
I20260709 13:03:44.071830 15237 rpcz_store.cc:276] 0709 13:03:42.934348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.027475 (+1093127us) service_pool.cc:224] Handling call
0709 13:03:44.027774 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.043029 (+ 15255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.043259 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.073824 15237 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:44.074575 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1944) took 1118 ms. Trace:
I20260709 13:03:44.072294 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:44.075006 15237 rpcz_store.cc:276] 0709 13:03:42.956718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.073294 (+1116576us) service_pool.cc:224] Handling call
0709 13:03:44.073635 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.074327 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.074518 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.025439 18341 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:44.065685 17831 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.075548 15841 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:44.039912 18264 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:44.082437 18354 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:44.082504 18157 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:44.082862 17867 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.083016 18157 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:44.080454 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:44.048611 18223 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.087954 15841 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:44.097240 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:44.067555 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101837us
I20260709 13:03:44.099728 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:44.103148 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:44.109385 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:44.110615 15841 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:44.111449 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:44.114409 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:44.119072 15157 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:44.077749 18242 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.043299 18243 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.077596 18150 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:44.084832 18251 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:44.150786 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.089027 17831 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:44.130877 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:44.188305 15841 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:44.113771 18171 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:44.083544 17867 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.115093 17986 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:44.127163 18242 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:44.093024 18223 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:44.204342 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:44.198181 18171 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.205699 18171 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:44.117054 17880 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:44.209342 17880 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:44.189689 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.062779 17631 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:44.205309 15841 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:44.175652 18359 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.179698 18227 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:44.179579 15154 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:44.296382 15157 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:44.299427 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 615) took 1061 ms. Trace:
I20260709 13:03:44.299966 15157 rpcz_store.cc:276] 0709 13:03:43.237718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.289462 (+1051744us) service_pool.cc:224] Handling call
0709 13:03:44.296032 (+  6570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.299105 (+  3073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.299350 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:44.104295 18157 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.316946 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:44.330420 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:44.331897 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:44.332643 15841 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
W20260709 13:03:44.334633 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.340981 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:44.342193 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:44.350843 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:44.307526 18361 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.348122 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 318223us
W20260709 13:03:44.370051 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 347208us
W20260709 13:03:44.357731 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 339305us
I20260709 13:03:44.323801 18227 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:44.393887 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2493) took 2780 ms (client timeout 1638 ms). Trace:
W20260709 13:03:44.394578 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.333956 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:44.396414 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89708.1us
W20260709 13:03:44.394543 15156 rpcz_store.cc:269] 0709 13:03:41.613630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.613757 (+   127us) service_pool.cc:224] Handling call
0709 13:03:44.393630 (+2779873us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:44.393813 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2008120064}
I20260709 13:03:44.398905 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:44.401137 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.401705 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.362895 18157 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:44.338131 18171 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.288864 18158 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:44.379709 18301 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:44.304328 18359 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.378839 18356 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:44.420421 15085 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:44.422000 18360 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:03:44.423605 18307 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.558s	user 0.018s	sys 0.008s
W20260709 13:03:44.426636 18265 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.009s
I20260709 13:03:44.380359 18361 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:44.431731 18274 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.977s	user 0.011s	sys 0.014s
W20260709 13:03:44.434334 18253 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.540s	user 0.016s	sys 0.008s
W20260709 13:03:44.435640 18306 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.098s	user 0.006s	sys 0.005s
W20260709 13:03:44.489609 18330 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.001s
I20260709 13:03:44.490154 18330 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.412748 18267 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:44.435732 18331 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.437381 17831 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.471997 18364 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:44.473418 18340 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:44.481950 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2487) took 3356 ms. Trace:
W20260709 13:03:44.450039 18363 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:44.432382 18233 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.483199 15845 raft_consensus.cc:2410] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:44.453513 18227 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.486327 18274 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:44.486928 18184 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.094s	user 0.010s	sys 0.006s
I20260709 13:03:44.449630 17867 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:44.492285 18154 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:44.500047 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 802) took 2356 ms (client timeout 2821 ms). Trace:
I20260709 13:03:44.454398 17631 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:44.504632 15162 rpcz_store.cc:276] 0709 13:03:41.125754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.327471 (+201717us) service_pool.cc:224] Handling call
0709 13:03:41.350376 (+ 22905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.567057 (+1216681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.479309 (+1912252us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400187ed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:44.481878 (+  2569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2773233536}
I20260709 13:03:44.520135 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1757) took 2702 ms. Trace:
I20260709 13:03:44.440811 18341 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:44.482131 18307 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:44.765547 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 205437us
I20260709 13:03:44.667511 15840 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:44.662989 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 214206us
I20260709 13:03:44.776669 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2507) took 1789 ms. Trace:
W20260709 13:03:44.662629 15814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.017s
I20260709 13:03:44.510617 18364 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:44.969983 18282 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.388s	user 0.017s	sys 0.011s
I20260709 13:03:44.951972 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1941) took 2927 ms. Trace:
W20260709 13:03:44.806655 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.806586 15159 rpcz_store.cc:276] 0709 13:03:42.987152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.700650 (+713498us) service_pool.cc:224] Handling call
0709 13:03:43.832632 (+131982us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:43.832983 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:43.890055 (+ 57072us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.729145 (+839090us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:44.755603 (+ 26458us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.776259 (+ 20656us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:44.776305 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.776377 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.776422 (+    45us) raft_consensus.cc:1685] finished
0709 13:03:44.776434 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.776591 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5500,"prepare.queue_time_us":54781,"prepare.run_cpu_time_us":291,"prepare.run_wall_time_us":1833,"raft.queue_time_us":144350,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":19619,"spinlock_wait_cycles":1836288,"thread_start_us":721341,"threads_started":2,"wal-append.queue_time_us":482769}
W20260709 13:03:44.729912 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 319434us
I20260709 13:03:44.604969 15847 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8.
W20260709 13:03:45.123205 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.802953 17882 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:44.803462 18215 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:45.141192 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:44.801301 18121 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:45.142074 18264 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.036s	user 0.010s	sys 0.000s
I20260709 13:03:45.142499 18264 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.143285 18378 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:45.143512 18331 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:44.889916 18275 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.920s	user 0.009s	sys 0.007s
I20260709 13:03:44.799708 18171 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:45.144976 18223 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.848s	user 0.011s	sys 0.001s
I20260709 13:03:45.145437 18223 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.147785 18223 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:45.148103 18359 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:44.510856 18340 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:44.765888 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2508) took 1754 ms. Trace:
W20260709 13:03:45.122066 15814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.019s	sys 0.020s
W20260709 13:03:45.138116 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1945) took 3112 ms (client timeout 1777 ms). Trace:
W20260709 13:03:45.159179 15161 rpcz_store.cc:269] 0709 13:03:42.025929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.097394 (+ 71465us) service_pool.cc:224] Handling call
0709 13:03:45.124457 (+3027063us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540017f054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.137992 (+ 13535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580755840}
I20260709 13:03:45.158293 15158 rpcz_store.cc:276] 0709 13:03:43.011662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.953914 (+942252us) service_pool.cc:224] Handling call
0709 13:03:43.993767 (+ 39853us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:43.994327 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.994370 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:43.995155 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.088724 (+ 93569us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:44.104687 (+ 15963us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.178942 (+ 74255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.179034 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.765605 (+586571us) raft_consensus.cc:1685] finished
0709 13:03:44.765623 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.765785 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24715,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":153,"raft.queue_time_us":323767,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":238,"thread_start_us":166994,"threads_started":2,"wal-append.queue_time_us":345005}
I20260709 13:03:45.160264 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:45.161000 15161 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:44.806051 15084 rpcz_store.cc:276] 0709 13:03:41.818168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.865947 (+ 47779us) service_pool.cc:224] Handling call
0709 13:03:41.870635 (+  4688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.906763 (+1036128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.519802 (+1613039us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:44.520050 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2366781696}
I20260709 13:03:45.160521 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:44.533875 15081 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:03:45.138896 15079 rpcz_store.cc:269] 0709 13:03:42.143462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.147589 (+  4127us) service_pool.cc:224] Handling call
0709 13:03:44.499821 (+2352232us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400307354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:44.499969 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731418368}
W20260709 13:03:44.952893 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1786) took 1903 ms (client timeout 1840 ms). Trace:
I20260709 13:03:44.953465 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 886) took 1464 ms. Trace:
I20260709 13:03:45.107424 15163 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:45.180008 18242 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.903s	user 0.009s	sys 0.001s
I20260709 13:03:44.740315 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1939) took 2686 ms. Trace:
I20260709 13:03:45.142576 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:45.181171 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:45.181298 15844 rpcz_store.cc:276] 0709 13:03:43.489563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.786938 (+297375us) service_pool.cc:224] Handling call
0709 13:03:44.953326 (+1166388us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400809654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:44.953415 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326082944}
I20260709 13:03:45.182531 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:45.123289 15231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.023s	sys 0.047s
W20260709 13:03:45.183276 15231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.023s	sys 0.047s
W20260709 13:03:45.163364 15233 rpcz_store.cc:269] 0709 13:03:43.049407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.083160 (+ 33753us) service_pool.cc:224] Handling call
0709 13:03:44.952683 (+1869523us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54000abed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:44.952805 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704065920}
I20260709 13:03:45.184680 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 633) took 1722 ms. Trace:
I20260709 13:03:44.961110 18311 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:03:44.561928 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.933732 18233 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:45.099123 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 209226us
I20260709 13:03:45.122834 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:45.152643 15155 rpcz_store.cc:276] 0709 13:03:42.024631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.024751 (+   120us) service_pool.cc:224] Handling call
0709 13:03:43.824059 (+1799308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.859763 (+ 35704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.951755 (+1091992us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540013f6d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:44.951895 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4137246080}
W20260709 13:03:45.219782 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.220338 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.124884 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1761) took 3186 ms (client timeout 2147 ms). Trace:
I20260709 13:03:44.601776 18265 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.752627 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 354473us
I20260709 13:03:45.021394 18245 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.185557 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:45.187021 15234 rpcz_store.cc:276] 0709 13:03:42.054112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.547398 (+493286us) service_pool.cc:224] Handling call
0709 13:03:42.554594 (+  7196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.625701 (+ 71107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.740061 (+2114360us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b54002d3a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:44.740238 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161879424}
I20260709 13:03:44.613231 18330 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:44.969686 18274 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:45.150892 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.192221 17867 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:45.192523 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 923) took 1847 ms (client timeout 2084 ms). Trace:
I20260709 13:03:45.193048 18377 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.186838 18331 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.164603 18215 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:44.923295 17831 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:45.199222 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:45.199589 18348 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.467s	user 0.000s	sys 0.002s
I20260709 13:03:45.140542 18364 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:45.201149 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:45.201783 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 897) took 1500 ms. Trace:
I20260709 13:03:45.203694 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1776) took 2325 ms. Trace:
I20260709 13:03:45.204491 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:45.205545 18355 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.883s	user 0.003s	sys 0.000s
I20260709 13:03:45.205837 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 617) took 1207 ms. Trace:
I20260709 13:03:45.206816 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bd17b479354abca0014ae059714e8f"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
W20260709 13:03:45.211355 18321 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.123s	user 0.010s	sys 0.014s
I20260709 13:03:45.144348 18275 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.213145 18104 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.213688 18374 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.187361 18153 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.214628 18379 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.215378 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1773) took 2342 ms. Trace:
I20260709 13:03:45.215484 18378 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:45.216183 18188 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:45.138700 18308 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.522653 18243 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:45.180456 18242 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.220012 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:45.164995 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:44.618660 18370 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.220762 15087 rpcz_store.cc:269] 0709 13:03:41.938953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.015996 (+ 77043us) service_pool.cc:224] Handling call
0709 13:03:45.124696 (+3108700us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.124799 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694087424}
I20260709 13:03:45.222713 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:45.222882 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:45.226094 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.227703 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:45.235270 18359 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:45.183162 18171 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:03:45.245949 15840 rpcz_store.cc:269] 0709 13:03:43.345209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.355092 (+  9883us) service_pool.cc:224] Handling call
0709 13:03:45.192357 (+1837265us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.192461 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600934656}
I20260709 13:03:45.194718 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:44.670908 18369 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.575s
I20260709 13:03:45.189213 18311 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:45.189738 18154 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:45.162977 18282 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:44.618507 18356 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.748306 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:45.185106 15231 rpcz_store.cc:276] 0709 13:03:43.462053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.504169 (+1042116us) service_pool.cc:224] Handling call
0709 13:03:44.504498 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.504723 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:44.518221 (+ 13498us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.184123 (+665902us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.184325 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:45.184366 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.184415 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.184499 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.184627 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26781,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1003,"thread_start_us":665319,"threads_started":1}
I20260709 13:03:46.069372 15844 rpcz_store.cc:276] 0709 13:03:43.701412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.182310 (+1480898us) service_pool.cc:224] Handling call
0709 13:03:45.201717 (+ 19407us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:45.990015 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.992975 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1784) took 2946 ms (client timeout 1987 ms). Trace:
W20260709 13:03:45.990477 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 878) took 2506 ms (client timeout 2287 ms). Trace:
W20260709 13:03:45.569038 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.110002 15236 rpcz_store.cc:276] 0709 13:03:42.872495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.317379 (+444884us) service_pool.cc:224] Handling call
0709 13:03:45.215183 (+1897804us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400318654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.215316 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237147392}
W20260709 13:03:46.112905 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.442305 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.093195 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2510) took 2310 ms. Trace:
W20260709 13:03:45.989050 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 803518us
I20260709 13:03:46.113703 15228 raft_consensus.cc:2410] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:45.288318 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:46.093657 15847 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 938) took 2705 ms (client timeout 2056 ms). Trace:
W20260709 13:03:45.714265 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.110052 15155 raft_consensus.cc:2376] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7. Re-sending same reply.
W20260709 13:03:45.988821 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 764292us
I20260709 13:03:45.478448 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1778) took 1172 ms. Trace:
W20260709 13:03:46.110626 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 653) took 1309 ms (client timeout 1571 ms). Trace:
W20260709 13:03:46.136758 15233 rpcz_store.cc:269] 0709 13:03:44.800749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.184375 (+383626us) service_pool.cc:224] Handling call
0709 13:03:46.110522 (+926147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.114696 15234 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:46.121760 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.74801e+06us
W20260709 13:03:46.121228 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1962) took 1588 ms (client timeout 1807 ms). Trace:
W20260709 13:03:46.114956 15847 rpcz_store.cc:269] 0709 13:03:43.388723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.418546 (+ 29823us) service_pool.cc:224] Handling call
0709 13:03:45.479986 (+2061440us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54006d4054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.093572 (+613586us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724294528}
W20260709 13:03:46.070974 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.139380 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:46.139027 15162 rpcz_store.cc:269] 0709 13:03:44.532989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.169487 (+636498us) service_pool.cc:224] Handling call
0709 13:03:46.121171 (+951684us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.069124 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 1948) took 3076 ms. Trace:
W20260709 13:03:46.120114 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1965) took 1493 ms (client timeout 1840 ms). Trace:
W20260709 13:03:46.140740 15155 rpcz_store.cc:269] 0709 13:03:44.626980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.219655 (+592675us) service_pool.cc:224] Handling call
0709 13:03:46.120034 (+900379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.141674 15841 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:46.113554 15154 rpcz_store.cc:276] 0709 13:03:43.782449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.301301 (+518852us) service_pool.cc:224] Handling call
0709 13:03:44.301669 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.514356 (+212687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.514396 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.579357 (+ 64961us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.755499 (+176142us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:44.755549 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.755642 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.092882 (+1337240us) raft_consensus.cc:1685] finished
0709 13:03:46.092912 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.093129 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125056,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":48,"thread_start_us":353476,"threads_started":2,"wal-append.queue_time_us":678276}
W20260709 13:03:46.141912 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1960) took 1657 ms (client timeout 1715 ms). Trace:
I20260709 13:03:46.140472 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:46.111353 15161 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:03:46.143409 15158 rpcz_store.cc:269] 0709 13:03:44.484532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.160009 (+675477us) service_pool.cc:224] Handling call
0709 13:03:46.141849 (+981840us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:46.144084 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2520) took 1678 ms (client timeout 1815 ms). Trace:
I20260709 13:03:46.110606 15159 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:46.109673 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.070776 15846 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:46.157640 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:46.114364 15202 connection.cc:256] RPC call timeout handler was delayed by -1.0294897490000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:03:46.144448 15161 rpcz_store.cc:269] 0709 13:03:44.465357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.164706 (+699349us) service_pool.cc:224] Handling call
0709 13:03:46.144027 (+979321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.180920 18131 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.181268 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:45.238930 18265 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.183274 18265 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.183799 18265 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.184242 18265 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.249624 18245 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.187053 18245 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.185289 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 924) took 2839 ms. Trace:
I20260709 13:03:46.187758 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:46.188249 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.188468 15233 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.190178 18275 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:45.970759 18157 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.557s	user 0.011s	sys 0.002s
I20260709 13:03:45.254325 18307 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:45.448809 18352 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.494s	user 0.003s	sys 0.000s
I20260709 13:03:46.192909 15084 raft_consensus.cc:606] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:46.245349 15084 raft_consensus.cc:614] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:45.398166 18341 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:46.144680 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 940201us
I20260709 13:03:46.193177 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:46.158314 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:46.110855 15232 rpcz_store.cc:269] 0709 13:03:43.042271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.103537 (+ 61266us) service_pool.cc:224] Handling call
0709 13:03:45.423234 (+2319697us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54003d9fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.988786 (+565552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714606464}
I20260709 13:03:46.158394 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 651) took 1648 ms. Trace:
W20260709 13:03:45.985996 18384 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.571s	user 0.000s	sys 0.001s
I20260709 13:03:46.114097 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.109856 15229 rpcz_store.cc:276] 0709 13:03:42.878415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.012665 (+1134250us) service_pool.cc:224] Handling call
0709 13:03:44.013048 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.014436 (+  1388us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:44.353186 (+338750us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.806824 (+453638us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.807387 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.807553 (+   166us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:44.807598 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.807685 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.203483 (+395798us) raft_consensus.cc:1685] finished
0709 13:03:45.203504 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.203634 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36654,"prepare.queue_time_us":189571,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":91749,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":69866,"thread_start_us":620730,"threads_started":2,"wal-append.queue_time_us":820589}
W20260709 13:03:46.140102 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.153733 18378 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:45.307300 15839 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:46.196713 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:46.264595 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1967) took 1073 ms. Trace:
I20260709 13:03:46.192337 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 903) took 1037 ms. Trace:
I20260709 13:03:46.151471 17831 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:46.264919 15161 rpcz_store.cc:276] 0709 13:03:45.191568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.196534 (+1004966us) service_pool.cc:224] Handling call
0709 13:03:46.264534 (+ 68000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.140580 15237 rpcz_store.cc:276] 0709 13:03:42.992434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.076418 (+1083984us) service_pool.cc:224] Handling call
0709 13:03:44.076770 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:44.095003 (+ 18233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.095051 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:44.114537 (+ 19486us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.560789 (+446252us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:44.560868 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.560975 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.068909 (+1507934us) raft_consensus.cc:1685] finished
0709 13:03:46.068934 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.069068 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16965,"raft.queue_time_us":27465,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":33767,"thread_start_us":366364,"threads_started":1,"wal-append.queue_time_us":1073898}
I20260709 13:03:46.265156 15844 rpcz_store.cc:276] 0709 13:03:45.154711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.152744 (+998033us) service_pool.cc:224] Handling call
0709 13:03:46.192279 (+ 39535us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:45.563791 18255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.005s	sys 0.013s
I20260709 13:03:46.192111 18223 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:03:46.232893 15054 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.000s	sys 0.022s
I20260709 13:03:46.067337 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 921) took 2722 ms. Trace:
W20260709 13:03:46.241146 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 882) took 2753 ms (client timeout 2718 ms). Trace:
I20260709 13:03:46.270476 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:45.422178 18339 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:46.238102 15233 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:46.272126 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 656) took 1018 ms. Trace:
I20260709 13:03:46.187573 15842 rpcz_store.cc:276] 0709 13:03:43.345492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.357939 (+ 12447us) service_pool.cc:224] Handling call
0709 13:03:43.358458 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.358774 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.358815 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.388583 (+ 29768us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.436346 (+ 47763us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:43.436417 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.436526 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.198991 (+1762465us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:46.184967 (+985976us) raft_consensus.cc:1685] finished
0709 13:03:46.184995 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.185227 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35238144,"thread_start_us":47294,"threads_started":1,"wal-append.queue_time_us":178523}
W20260709 13:03:46.111961 15845 rpcz_store.cc:269] 0709 13:03:43.484038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.484404 (+   366us) service_pool.cc:224] Handling call
0709 13:03:45.961284 (+2476880us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b540036db54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:45.990386 (+ 29102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2063810944}
I20260709 13:03:46.273568 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.274650 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 848) took 3020 ms. Trace:
I20260709 13:03:46.274959 15848 rpcz_store.cc:276] 0709 13:03:43.254177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.257147 (+  2970us) service_pool.cc:224] Handling call
0709 13:03:43.270209 (+ 13062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.326695 (+ 56486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.326747 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.359520 (+ 32773us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.021755 (+662235us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:44.021815 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.021911 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.204992 (+1183081us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:46.269297 (+1064305us) raft_consensus.cc:1685] finished
0709 13:03:46.269320 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.274600 (+  5280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55888,"raft.queue_time_us":59974,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":118,"spinlock_wait_cycles":32398080,"thread_start_us":597141,"threads_started":1,"wal-append.queue_time_us":714048}
W20260709 13:03:46.144909 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1964) took 1529 ms (client timeout 1777 ms). Trace:
W20260709 13:03:46.242017 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101607us
I20260709 13:03:45.372304 18121 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.070181 15156 rpcz_store.cc:276] 0709 13:03:43.998348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.411587 (+413239us) service_pool.cc:224] Handling call
0709 13:03:44.412173 (+   586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.453744 (+ 41571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.453793 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.454453 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.733576 (+279123us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:44.819997 (+ 86421us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.125069 (+305072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.125147 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.205631 (+ 80484us) raft_consensus.cc:1685] finished
0709 13:03:45.205651 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.205786 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109197,"prepare.queue_time_us":83058,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":397753,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":156,"spinlock_wait_cycles":1942272,"thread_start_us":278667,"threads_started":1,"wal-append.queue_time_us":749091}
I20260709 13:03:45.403157 18108 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.243808 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:46.279309 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.279676 15158 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.264972 17880 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.052830 18401 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.374733 18266 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.187211 18265 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.246591 15236 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:46.141321 18184 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:46.248994 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.260898 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:46.261057 15228 rpcz_store.cc:276] 0709 13:03:44.510255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.403933 (+893678us) service_pool.cc:224] Handling call
0709 13:03:46.158322 (+754389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.261612 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.261773 15155 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:46.262288 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 148272us
I20260709 13:03:46.288458 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1775) took 3415 ms. Trace:
W20260709 13:03:46.288746 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1963) took 1686 ms (client timeout 2074 ms). Trace:
I20260709 13:03:46.289005 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:03:46.289256 15155 rpcz_store.cc:269] 0709 13:03:44.602966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.192930 (+1589964us) service_pool.cc:224] Handling call
0709 13:03:46.288693 (+ 95763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.290387 15155 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:46.290962 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 621) took 2251 ms. Trace:
W20260709 13:03:46.291507 15201 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.718s (SENT)
I20260709 13:03:46.285355 17880 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:46.196121 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52009.7us
I20260709 13:03:46.271931 18339 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:46.263175 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 655) took 1040 ms. Trace:
I20260709 13:03:46.297555 15231 rpcz_store.cc:276] 0709 13:03:45.223045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.109650 (+886605us) service_pool.cc:224] Handling call
0709 13:03:46.263103 (+153453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.140189 15081 rpcz_store.cc:276] 0709 13:03:44.305805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.360907 (+ 55102us) service_pool.cc:224] Handling call
0709 13:03:44.386226 (+ 25319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.221421 (+835195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.478010 (+256589us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003250d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:45.478346 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338722688}
I20260709 13:03:46.241150 18243 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.266067 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:46.317696 15237 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.317812 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.181288 18274 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:46.118505 18154 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.318760 15848 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
W20260709 13:03:46.197274 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 894) took 2591 ms (client timeout 2716 ms). Trace:
I20260709 13:03:46.101046 18321 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.283249 18331 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.246402 18237 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:46.100806 18153 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.115571 18215 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:45.671643 18269 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:46.270800 15054 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.000s	sys 0.024s
W20260709 13:03:46.334973 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 846241us
I20260709 13:03:46.138079 18379 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:46.270975 15846 rpcz_store.cc:269] 0709 13:03:43.487556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.747944 (+2260388us) service_pool.cc:224] Handling call
0709 13:03:46.241067 (+493123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.317950 18266 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.272470 15233 rpcz_store.cc:276] 0709 13:03:45.253696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.143015 (+889319us) service_pool.cc:224] Handling call
0709 13:03:46.272072 (+129057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.282708 18121 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.347177 15233 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:46.347991 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 646) took 2419 ms. Trace:
I20260709 13:03:46.340512 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:46.352965 15846 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:46.276276 15159 rpcz_store.cc:269] 0709 13:03:44.615518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.198921 (+583403us) service_pool.cc:224] Handling call
0709 13:03:46.144835 (+945914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.100607 18104 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.191939 17882 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
W20260709 13:03:46.277921 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 859) took 2214 ms (client timeout 2910 ms). Trace:
I20260709 13:03:46.276794 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:46.244359 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:46.359086 15842 rpcz_store.cc:269] 0709 13:03:44.063170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.273343 (+2210173us) service_pool.cc:224] Handling call
0709 13:03:46.277863 (+  4520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.151309 18308 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:46.271160 18255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.530s	user 0.013s	sys 0.029s
I20260709 13:03:46.281895 15156 raft_consensus.cc:1217] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Deduplicated request from leader. Original: 4.7->[5.8-7.9]   Dedup: 5.8->[7.9-7.9]
I20260709 13:03:45.257205 18377 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.284191 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.267266 17831 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.363710 15847 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:46.363804 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1958) took 1890 ms (client timeout 2185 ms). Trace:
I20260709 13:03:46.363983 15160 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:46.366110 15162 rpcz_store.cc:269] 0709 13:03:44.473538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.180773 (+1707235us) service_pool.cc:224] Handling call
0709 13:03:46.363737 (+182964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.272081 18332 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.262367 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.368949 15232 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:46.369653 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 379940us
I20260709 13:03:46.291265 15155 rpcz_store.cc:276] 0709 13:03:44.039479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.289953 (+2250474us) service_pool.cc:224] Handling call
0709 13:03:46.290231 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.290725 (+   494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.290913 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.137954 18374 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.115631 18370 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.294878 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 640) took 2824 ms. Trace:
I20260709 13:03:46.288889 15230 rpcz_store.cc:276] 0709 13:03:42.873046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.434406 (+561360us) service_pool.cc:224] Handling call
0709 13:03:43.434856 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.435346 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.435392 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.439383 (+  3991us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.576228 (+136845us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:44.011421 (+435193us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.011669 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.011759 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.422903 (+1411144us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:46.288183 (+865280us) raft_consensus.cc:1685] finished
0709 13:03:46.288270 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.288401 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31683,"prepare.queue_time_us":603657,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":5055,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":26304,"spinlock_wait_cycles":7842176,"thread_start_us":274104,"threads_started":2,"wal-append.queue_time_us":413743}
I20260709 13:03:46.378077 15236 rpcz_store.cc:276] 0709 13:03:43.470112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.180560 (+2710448us) service_pool.cc:224] Handling call
0709 13:03:46.246358 (+ 65798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.294605 (+ 48247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.294780 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.297602 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 627) took 1106 ms. Trace:
I20260709 13:03:46.379078 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 648) took 2413 ms. Trace:
I20260709 13:03:46.379385 15154 rpcz_store.cc:276] 0709 13:03:45.190698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.197130 (+1006432us) service_pool.cc:224] Handling call
0709 13:03:46.297519 (+100389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.339845 18242 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.240953 18359 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.310560 15234 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:03:46.383999 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 654) took 1262 ms (client timeout 1603 ms). Trace:
W20260709 13:03:46.384387 15234 rpcz_store.cc:269] 0709 13:03:45.121616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.225257 (+103641us) service_pool.cc:224] Handling call
0709 13:03:46.383941 (+1158684us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.342483 18237 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:46.315133 15158 raft_consensus.cc:2410] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:03:46.320499 15840 rpcz_store.cc:269] 0709 13:03:43.606455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.109408 (+2502953us) service_pool.cc:224] Handling call
0709 13:03:46.197220 (+ 87812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.325083 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.391250 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.358835 18399 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.141193 18397 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.377162 18332 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.401300 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:46.329954 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.330147 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:46.296979 18227 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.331663 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.304924 18188 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:46.335664 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:46.418341 15080 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.271343 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 625) took 1081 ms. Trace:
I20260709 13:03:46.341904 15157 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:46.348381 15233 rpcz_store.cc:276] 0709 13:03:43.928117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.346546 (+2418429us) service_pool.cc:224] Handling call
0709 13:03:46.346944 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.347667 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.347920 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.272352 18311 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.426700 18311 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:46.266335 18235 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:46.360085 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.327667 18243 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.363483 15156 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [5.8-7.9]
I20260709 13:03:46.367956 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:46.283733 18108 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:46.371495 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 94780.7us
I20260709 13:03:46.372118 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:46.437120 15085 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.326861 18184 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:03:46.373332 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:46.363978 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 905) took 1208 ms. Trace:
I20260709 13:03:46.439569 15844 rpcz_store.cc:276] 0709 13:03:45.155264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.274044 (+1118780us) service_pool.cc:224] Handling call
0709 13:03:46.363924 (+ 89880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.370101 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 647) took 2404 ms. Trace:
W20260709 13:03:46.439935 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 66721.8us
I20260709 13:03:46.441269 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 620) took 2405 ms. Trace:
I20260709 13:03:46.441537 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.441640 15156 rpcz_store.cc:276] 0709 13:03:44.035573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.281514 (+2245941us) service_pool.cc:224] Handling call
0709 13:03:46.281774 (+   260us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:46.441025 (+159251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.441217 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.440335 15232 rpcz_store.cc:276] 0709 13:03:43.965459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.369581 (+2404122us) service_pool.cc:224] Handling call
0709 13:03:46.369788 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.369900 (+   112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.369915 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.369940 (+    25us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:46.369958 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.369989 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.370064 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:46.374909 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 260485us
I20260709 13:03:46.298759 15228 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:03:46.444029 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 643) took 2973 ms. Trace:
I20260709 13:03:46.379472 15230 rpcz_store.cc:276] 0709 13:03:43.965815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.378322 (+2412507us) service_pool.cc:224] Handling call
0709 13:03:46.378604 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.378773 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.378796 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.378835 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:46.378877 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.378924 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.379028 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.444522 15228 rpcz_store.cc:276] 0709 13:03:43.470997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.298087 (+2827090us) service_pool.cc:224] Handling call
0709 13:03:46.298526 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.443688 (+145162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.443961 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.385517 15234 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:46.446024 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 650) took 1938 ms. Trace:
I20260709 13:03:46.446357 15234 rpcz_store.cc:276] 0709 13:03:44.507915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.385071 (+1877156us) service_pool.cc:224] Handling call
0709 13:03:46.385349 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.445756 (+ 60407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.445977 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.320240 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 909) took 1122 ms. Trace:
I20260709 13:03:46.447392 15848 rpcz_store.cc:276] 0709 13:03:45.197498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.284598 (+1087100us) service_pool.cc:224] Handling call
0709 13:03:46.320168 (+ 35570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.328472 18274 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.393203 15236 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:46.454898 15080 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:03:46.287985 18401 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.327214 15231 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:46.368825 17831 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.397341 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.400162 15847 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:46.403230 15083 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:46.403779 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:46.458770 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 645) took 2531 ms. Trace:
W20260709 13:03:46.459337 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 55633.7us
I20260709 13:03:46.459120 15231 rpcz_store.cc:276] 0709 13:03:43.927758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.318994 (+2391236us) service_pool.cc:224] Handling call
0709 13:03:46.319372 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.458538 (+139166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.458719 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.403759 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:46.405586 15159 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:46.420014 15161 rpcz_store.cc:276] 0709 13:03:45.190113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.270215 (+1080102us) service_pool.cc:224] Handling call
0709 13:03:46.271256 (+  1041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.271737 15843 rpcz_store.cc:276] 0709 13:03:43.344637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.347347 (+  2710us) service_pool.cc:224] Handling call
0709 13:03:43.377978 (+ 30631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.378478 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.378512 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.179812 (+1801300us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b2c000fb978. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:45.180274 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.653718 (+473444us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:45.672458 (+ 18740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.752572 (+ 80114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.066976 (+314404us) raft_consensus.cc:1685] finished
0709 13:03:46.066996 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.067251 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":20817152,"thread_start_us":262020,"threads_started":1,"wal-append.queue_time_us":885089}
I20260709 13:03:46.460632 15082 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:46.349112 18269 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.361361 18266 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.383787 18223 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:46.405223 18399 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.437458 15233 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:46.437280 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:46.438139 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1950) took 2058 ms. Trace:
I20260709 13:03:46.474503 18410 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:46.435513 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 858) took 2388 ms. Trace:
W20260709 13:03:46.393641 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 626) took 1203 ms (client timeout 1571 ms). Trace:
I20260709 13:03:46.480345 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:46.450791 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.418049 18188 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:46.417552 18044 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:46.448549 17914 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:46.463307 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.504339 15155 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:46.507294 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:46.508121 15156 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:46.428707 18227 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.513741 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:46.474285 15085 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:46.433324 18235 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:46.432194 15079 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:46.457027 18330 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:03:46.474776 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:46.436913 18282 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.479446 15154 rpcz_store.cc:276] 0709 13:03:44.379587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.407835 (+2028248us) service_pool.cc:224] Handling call
0709 13:03:46.437638 (+ 29803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.437801 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.437824 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.437862 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:46.437898 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.437985 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.438082 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:46.481048 15158 rpcz_store.cc:269] 0709 13:03:45.190429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.144231 (+953802us) service_pool.cc:224] Handling call
0709 13:03:46.393583 (+249352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.456292 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 649) took 1949 ms. Trace:
I20260709 13:03:46.523403 15236 rpcz_store.cc:276] 0709 13:03:44.506438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.379127 (+1872689us) service_pool.cc:224] Handling call
0709 13:03:46.392960 (+ 13833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.455126 (+ 62166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.456230 (+  1104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.479457 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 652) took 1966 ms. Trace:
I20260709 13:03:46.480464 15842 rpcz_store.cc:276] 0709 13:03:44.046782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.359831 (+2313049us) service_pool.cc:224] Handling call
0709 13:03:46.435448 (+ 75617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.525437 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:46.450344 18381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.013s	sys 0.023s
I20260709 13:03:46.379869 18339 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.550423 18339 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.550705 18215 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.551602 18215 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:46.552093 18215 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.448379 18341 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.460893 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 624) took 2367 ms. Trace:
I20260709 13:03:46.553508 15159 rpcz_store.cc:276] 0709 13:03:44.093790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.381844 (+2288054us) service_pool.cc:224] Handling call
0709 13:03:46.396202 (+ 14358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.460610 (+ 64408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.460795 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.554021 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.488751 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:46.555752 15844 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.420938 18397 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.510342 18308 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.461776 18171 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:03:46.475096 17882 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:46.493575 18255 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:46.494019 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1943) took 4468 ms. Trace:
I20260709 13:03:46.494462 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.561300 18255 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.561587 15083 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:46.562692 15235 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:46.563228 18065 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.563658 18255 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.495011 18237 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.563678 18065 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.497210 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.565186 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 902) took 1410 ms. Trace:
I20260709 13:03:46.565519 15843 rpcz_store.cc:276] 0709 13:03:45.154404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.477673 (+1323269us) service_pool.cc:224] Handling call
0709 13:03:46.565123 (+ 87450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.566247 18154 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.566392 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.487785 18242 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:46.567109 15843 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:46.449170 18378 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.390882 18359 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.518347 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.574096 15085 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:46.426728 18121 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.521549 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.488032 18044 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:46.524322 15233 rpcz_store.cc:276] 0709 13:03:44.512964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.420393 (+1907429us) service_pool.cc:224] Handling call
0709 13:03:46.437138 (+ 16745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.479177 (+ 42039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.479392 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.528528 15842 raft_consensus.cc:2410] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:03:46.502802 18223 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:46.537495 18377 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:46.569389 18396 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:46.540309 18401 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.545533 18374 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.520987 18235 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.549096 18331 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.439688 18364 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.592229 18044 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.483022 18410 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:46.552584 18215 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:46.521701 18400 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.225s	user 0.003s	sys 0.000s
I20260709 13:03:46.564496 18104 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.489307 18188 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.568262 15846 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:46.619225 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 907) took 1455 ms. Trace:
I20260709 13:03:46.619614 15846 rpcz_store.cc:276] 0709 13:03:45.164143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.340221 (+1176078us) service_pool.cc:224] Handling call
0709 13:03:46.619159 (+278938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.620579 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.621181 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 904) took 1466 ms. Trace:
I20260709 13:03:46.621364 15846 rpcz_store.cc:276] 0709 13:03:45.155012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.620291 (+1465279us) service_pool.cc:224] Handling call
0709 13:03:46.621144 (+   853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.621959 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.622445 15846 raft_consensus.cc:2410] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:46.622985 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.577322 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 857) took 2533 ms. Trace:
I20260709 13:03:46.580281 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:46.553115 18341 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.538707 18370 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.601924 15844 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:46.627182 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 908) took 1439 ms. Trace:
W20260709 13:03:46.627306 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47147.7us
I20260709 13:03:46.627666 15844 rpcz_store.cc:276] 0709 13:03:45.187814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.465968 (+1278154us) service_pool.cc:224] Handling call
0709 13:03:46.627101 (+161133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.628144 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:46.537495 18243 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.603407 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 861) took 2524 ms. Trace:
I20260709 13:03:46.629070 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:46.629168 15842 rpcz_store.cc:276] 0709 13:03:44.078667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.525182 (+2446515us) service_pool.cc:224] Handling call
0709 13:03:46.603306 (+ 78124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.629221 15844 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:46.587399 18131 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:03:46.529989 18381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.013s	sys 0.024s
I20260709 13:03:46.585454 18308 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.595196 18153 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.623592 15843 rpcz_store.cc:276] 0709 13:03:44.043755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.566123 (+2522368us) service_pool.cc:224] Handling call
0709 13:03:46.577265 (+ 11142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.630376 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 880) took 3144 ms. Trace:
W20260709 13:03:46.557719 18411 consensus_queue.cc:1175] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 6 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:46.636198 18223 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.588812 18378 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.638864 18378 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.640604 18397 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.641264 18374 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:46.641678 18374 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.642005 18374 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.642484 18308 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.643983 18104 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:46.645231 18327 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.646342 18269 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.650095 15843 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:03:46.651111 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 888) took 3160 ms. Trace:
I20260709 13:03:46.651687 15843 rpcz_store.cc:276] 0709 13:03:43.490227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.649277 (+3159050us) service_pool.cc:224] Handling call
0709 13:03:46.649778 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.650713 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.651008 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.635573 18412 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:46.632817 18379 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.653645 15843 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:46.654399 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 889) took 3163 ms. Trace:
I20260709 13:03:46.656049 18361 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.656349 18245 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:46.656920 18245 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:46.657878 18245 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.659723 18340 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.667316 18340 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.663048 18108 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.614530 18410 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:46.664150 18321 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.664757 18156 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:46.661527 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 937) took 3273 ms (client timeout 1880 ms). Trace:
I20260709 13:03:46.665189 18356 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:46.666352 18399 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.667553 18364 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.666043 18274 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:03:46.625661 18265 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.677167 18131 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:46.669658 18266 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.712729 18227 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.713621 18308 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.713845 18359 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:46.686434 18245 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.696573 18265 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.718977 18235 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.721282 18341 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.722826 18235 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.675233 18356 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:46.727593 18356 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.729048 18356 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:46.776211 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.783138 15085 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:46.605527 15154 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:46.786105 15088 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:46.786908 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1951) took 2402 ms. Trace:
I20260709 13:03:46.589769 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:46.788877 15158 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.789471 15159 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:46.790967 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1952) took 2403 ms. Trace:
I20260709 13:03:46.791520 15159 rpcz_store.cc:276] 0709 13:03:44.387492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.594155 (+2206663us) service_pool.cc:224] Handling call
0709 13:03:46.611846 (+ 17691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.790350 (+178504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.790880 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.561223 15163 rpcz_store.cc:276] 0709 13:03:42.025270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.305181 (+279911us) service_pool.cc:224] Handling call
0709 13:03:44.395540 (+2090359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:45.179013 (+783473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.926196 (+747183us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:46.493807 (+567611us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400378550. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.493962 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6203046528}
W20260709 13:03:46.607848 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133135us
I20260709 13:03:46.794850 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.795879 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1953) took 2395 ms. Trace:
I20260709 13:03:46.585477 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:46.797839 15156 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:46.603713 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.796447 15163 rpcz_store.cc:276] 0709 13:03:44.400814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.793943 (+2393129us) service_pool.cc:224] Handling call
0709 13:03:46.794828 (+   885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.795273 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.795307 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.795377 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:46.795422 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.795493 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.795641 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.814905 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.816522 15083 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:46.776222 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.810793 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.811089 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.670193 15839 rpcz_store.cc:269] 0709 13:03:43.388418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.407830 (+ 19412us) service_pool.cc:224] Handling call
0709 13:03:46.661288 (+3253458us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b540074e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.661442 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4338386688}
I20260709 13:03:46.839730 15839 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:46.674582 15841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 860) took 2606 ms (client timeout 2049 ms). Trace:
W20260709 13:03:46.841243 15841 rpcz_store.cc:269] 0709 13:03:44.068663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.333670 (+265007us) service_pool.cc:224] Handling call
0709 13:03:46.674265 (+2340595us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b540041c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.674477 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4013636352}
I20260709 13:03:46.844395 15841 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:03:46.635644 15844 rpcz_store.cc:276] 0709 13:03:43.486966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.628599 (+3141633us) service_pool.cc:224] Handling call
0709 13:03:46.628981 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.630078 (+  1097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.630308 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.861698 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.862388 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:46.863761 15844 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:46.672075 15846 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:46.864674 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.807964 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.865974 15848 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:46.867508 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.868022 15848 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:46.857247 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 890) took 3340 ms. Trace:
I20260709 13:03:46.869431 15839 rpcz_store.cc:276] 0709 13:03:43.500097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.838216 (+3338119us) service_pool.cc:224] Handling call
0709 13:03:46.839118 (+   902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.840454 (+  1336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.840742 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.872314 15839 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:46.787832 15154 rpcz_store.cc:276] 0709 13:03:44.383750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.575906 (+2192156us) service_pool.cc:224] Handling call
0709 13:03:46.605165 (+ 29259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.785998 (+180833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.786568 (+   570us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:46.874320 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 638) took 3404 ms (client timeout 1661 ms). Trace:
I20260709 13:03:46.827926 18227 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:03:46.876116 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.876382 18227 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:46.875459 15235 rpcz_store.cc:269] 0709 13:03:43.469574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.742236 (+272662us) service_pool.cc:224] Handling call
0709 13:03:46.873650 (+3131414us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b5400318b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.873947 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6214937984}
I20260709 13:03:46.878453 18227 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.880249 18154 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:46.828226 15840 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:46.830514 18131 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.834256 15845 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 3.
I20260709 13:03:46.836616 18381 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.673154 15842 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:46.888068 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 883) took 3400 ms. Trace:
I20260709 13:03:46.888520 15842 rpcz_store.cc:276] 0709 13:03:43.487853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.630074 (+3142221us) service_pool.cc:224] Handling call
0709 13:03:46.672871 (+ 42797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.887487 (+214616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.887864 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.889799 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:46.890071 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:46.654788 15843 rpcz_store.cc:276] 0709 13:03:43.490571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.652728 (+3162157us) service_pool.cc:224] Handling call
0709 13:03:46.653191 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.654096 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.654334 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.892347 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.868695 15079 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:46.893641 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.894049 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.895193 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 853) took 2875 ms. Trace:
I20260709 13:03:46.895249 18359 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:46.895718 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:46.851567 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.896116 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.898376 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.899489 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.853632 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1954) took 2450 ms. Trace:
I20260709 13:03:46.900480 18359 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.901031 18359 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:46.901489 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.902730 15086 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:46.902995 15235 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:46.900692 15163 rpcz_store.cc:276] 0709 13:03:44.402973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.815308 (+2412335us) service_pool.cc:224] Handling call
0709 13:03:46.815825 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.816293 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.816317 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.816393 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:46.816421 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.816488 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.853554 (+ 37066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.857187 15841 pending_rounds.cc:85] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 8
I20260709 13:03:46.907080 15841 pending_rounds.cc:107] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:03:46.907492 15841 raft_consensus.cc:2889] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:46.907823 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1779) took 2601 ms. Trace:
I20260709 13:03:46.884258 18381 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:46.796486 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.901630 18264 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.881165 18154 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:46.904920 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:46.916910 18264 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:46.917732 18264 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:46.919482 18264 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:46.921341 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:46.922340 15236 raft_consensus.cc:2376] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8. Re-sending same reply.
I20260709 13:03:46.922763 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:46.853693 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.908282 15088 rpcz_store.cc:276] 0709 13:03:44.306095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.360085 (+ 53990us) service_pool.cc:224] Handling call
0709 13:03:44.500741 (+140656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.786947 (+2286206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.906368 (+119421us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400307354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:46.907534 (+  1166us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400307350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.907747 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5575909632}
I20260709 13:03:46.909286 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 900) took 2916 ms. Trace:
I20260709 13:03:46.855680 18339 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.892513 15842 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:46.913275 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 892) took 3339 ms. Trace:
I20260709 13:03:46.933838 15841 rpcz_store.cc:276] 0709 13:03:43.569127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.842773 (+3273646us) service_pool.cc:224] Handling call
0709 13:03:46.843465 (+   692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.908202 (+ 64737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.908430 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.913913 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.914161 18235 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:46.914351 17880 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:46.914726 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 118483us
I20260709 13:03:46.899878 15159 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:03:46.924700 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:46.895848 15843 rpcz_store.cc:276] 0709 13:03:44.020074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.893291 (+2873217us) service_pool.cc:224] Handling call
0709 13:03:46.893789 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.894548 (+   759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.894587 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.894668 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:46.894726 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.894804 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.894950 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6495,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":291}
I20260709 13:03:46.928889 18266 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:46.929209 18381 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:46.930306 15156 raft_consensus.cc:2410] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:46.926962 15848 rpcz_store.cc:276] 0709 13:03:43.964755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.866871 (+2902116us) service_pool.cc:224] Handling call
0709 13:03:46.867462 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.868586 (+  1124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.881531 (+ 12945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.936231 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 851) took 2906 ms. Trace:
I20260709 13:03:46.936764 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 852) took 2917 ms. Trace:
I20260709 13:03:46.942689 15844 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:46.943439 15842 rpcz_store.cc:276] 0709 13:03:44.019702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.891418 (+2871716us) service_pool.cc:224] Handling call
0709 13:03:46.891939 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.934849 (+ 42910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.936707 (+  1858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.943264 15839 rpcz_store.cc:276] 0709 13:03:44.019272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.870968 (+2851696us) service_pool.cc:224] Handling call
0709 13:03:46.871615 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.882246 (+ 10631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.925355 (+ 43109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.925040 17914 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:46.951345 18321 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.951753 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:46.917142 18154 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:47.057269 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:47.059217 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 136878us
W20260709 13:03:46.972575 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.050190 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.952360 18418 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:46.948287 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1955) took 2536 ms. Trace:
I20260709 13:03:46.973366 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.977229 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.980644 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34666 (request call id 24) took 1775 ms. Trace:
W20260709 13:03:46.971297 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 898) took 3178 ms (client timeout 3371 ms). Trace:
I20260709 13:03:47.066725 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.974519 18332 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:46.952776 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
W20260709 13:03:47.063740 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 111195us
I20260709 13:03:47.065260 15159 rpcz_store.cc:276] 0709 13:03:44.407268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.858062 (+2450794us) service_pool.cc:224] Handling call
0709 13:03:46.897151 (+ 39089us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:46.936886 (+ 39735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.943420 (+  6534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:46.936340 18235 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:47.059401 15128 connection.cc:256] RPC call timeout handler was delayed by -1.0173883789999536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 13:03:46.975953 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:46.952328 18321 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:47.072510 15843 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 3.
I20260709 13:03:46.951826 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:47.066808 15845 rpcz_store.cc:269] 0709 13:03:43.792715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.274134 (+2481419us) service_pool.cc:224] Handling call
0709 13:03:46.970913 (+696779us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54002a1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.971166 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273489792}
I20260709 13:03:46.977234 18311 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.075452 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.074003 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.091349 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:47.065979 15084 rpcz_store.cc:276] 0709 13:03:45.205344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.206690 (+  1346us) service_pool.cc:224] Handling call
0709 13:03:46.980094 (+1773404us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540014da54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:46.980322 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281110912}
I20260709 13:03:47.070643 18235 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:47.060022 17914 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.097124 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.106022 15086 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:47.107573 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:47.110787 18388 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.536s	user 0.008s	sys 0.007s
I20260709 13:03:47.111728 18065 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:47.113108 17831 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.008s
I20260709 13:03:47.113708 17831 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.118129 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 868) took 1896 ms (client timeout 2134 ms). Trace:
I20260709 13:03:47.118594 15081 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:47.120146 15842 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:47.120885 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 866) took 2974 ms. Trace:
I20260709 13:03:47.118153 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:47.121302 15842 rpcz_store.cc:276] 0709 13:03:44.146941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.119378 (+2972437us) service_pool.cc:224] Handling call
0709 13:03:47.119853 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.120600 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.120809 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:47.122843 18341 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:47.123816 15088 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
W20260709 13:03:47.124135 15810 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.661s (SENT)
I20260709 13:03:47.121064 17880 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.125238 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 623) took 3041 ms. Trace:
I20260709 13:03:47.125488 17880 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.125659 15155 rpcz_store.cc:276] 0709 13:03:44.083771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.370538 (+2286767us) service_pool.cc:224] Handling call
0709 13:03:46.370782 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.504735 (+133953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.124720 (+619985us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54001d7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.125036 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306695040}
I20260709 13:03:47.126495 15088 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:47.099578 18330 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:03:47.129172 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.004s	sys 0.017s
W20260709 13:03:47.129733 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 649452us
I20260709 13:03:47.132401 15052 leader_election.cc:400] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8.
W20260709 13:03:47.118626 15840 rpcz_store.cc:269] 0709 13:03:45.221733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.390983 (+1169250us) service_pool.cc:224] Handling call
0709 13:03:47.117892 (+726909us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400806454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:47.118063 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002631552}
I20260709 13:03:47.135066 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.135732 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 867) took 1914 ms. Trace:
I20260709 13:03:47.136010 15156 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:47.098062 15158 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:47.140202 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:47.120662 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.142490 18321 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:47.142855 15081 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:47.070741 18339 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.144395 15034 auto_rebalancer.cc:629] Failed to schedule move for tablet 38a7fd74046f492f8ef8366fbc675ee0: Illegal state: Replica 3d45101e0851484690b4f03a4dcdb63d is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:47.145355 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:47.138937 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:47.157330 15848 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:47.160554 18339 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:47.138049 18341 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:47.132726 15052 leader_election.cc:403] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:03:47.161916 15841 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.136900 15842 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:47.124671 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:47.136169 15840 rpcz_store.cc:276] 0709 13:03:45.221421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.134368 (+1912947us) service_pool.cc:224] Handling call
0709 13:03:47.134885 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.135249 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.135362 (+   113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:47.135443 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:47.135500 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.135558 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.135668 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:47.165143 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:46.981825 18369 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:47.179708 18369 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:47.180083 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.181318 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.182224 18381 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8.
I20260709 13:03:47.075031 18156 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.070333 18266 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.158552 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:47.177619 18044 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.010s
I20260709 13:03:47.177922 15840 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:47.178534 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.178728 17880 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:47.092255 18418 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.185361 15086 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:47.116564 17831 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.181170 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:47.188884 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48822.8us
W20260709 13:03:47.181416 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49349.5us
I20260709 13:03:46.951346 18188 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.210305 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 258311us
I20260709 13:03:47.214514 18044 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.215029 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.215068 18327 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:47.216984 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.217602 15839 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:47.217706 17914 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.219152 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2512) took 3435 ms. Trace:
I20260709 13:03:47.221742 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:47.191088 18154 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:47.222885 18370 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:47.224404 18215 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.012s
I20260709 13:03:47.187299 18359 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.233227 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 901) took 2718 ms. Trace:
W20260709 13:03:47.235980 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.238468 15841 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:47.188390 18330 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:47.193254 18410 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.009s
I20260709 13:03:47.236676 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.211776 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 869) took 1986 ms. Trace:
I20260709 13:03:47.267127 15840 rpcz_store.cc:276] 0709 13:03:45.225525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.177114 (+1951589us) service_pool.cc:224] Handling call
0709 13:03:47.177632 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.204743 (+ 27111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.211703 (+  6960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:47.189369 18243 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:47.248286 15157 rpcz_store.cc:276] 0709 13:03:43.783057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.316815 (+533758us) service_pool.cc:224] Handling call
0709 13:03:44.340006 (+ 23191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.420215 (+2080209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.218552 (+798337us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54000866d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.218885 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4603335552}
I20260709 13:03:47.259258 15233 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.261106 15842 rpcz_store.cc:276] 0709 13:03:44.515322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.122295 (+2606973us) service_pool.cc:224] Handling call
0709 13:03:47.122788 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.175024 (+ 52236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.232954 (+ 57930us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.233160 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13459200}
I20260709 13:03:47.275622 18266 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.277516 15160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 1956) took 2863 ms (client timeout 2201 ms). Trace:
W20260709 13:03:47.278106 15160 rpcz_store.cc:269] 0709 13:03:44.413615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.227373 (+813758us) service_pool.cc:224] Handling call
0709 13:03:47.247471 (+2020098us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:47.277239 (+ 29768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580055936}
I20260709 13:03:47.279013 18266 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.279544 18327 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.280184 18266 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:47.281126 15082 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:47.281167 15847 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 906) took 2093 ms (client timeout 1556 ms). Trace:
I20260709 13:03:47.247543 18339 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:47.181200 18369 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.165475 18311 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.282353 15847 rpcz_store.cc:269] 0709 13:03:45.158793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.140104 (+981311us) service_pool.cc:224] Handling call
0709 13:03:47.237930 (+1097826us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400498c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:47.252160 (+ 14230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390775680}
I20260709 13:03:47.236424 18156 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.294657 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:47.303399 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78374.1us
I20260709 13:03:47.309113 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.283325 18264 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:47.312083 18369 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.312307 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.313504 15163 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:47.316203 15085 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:47.266443 18330 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.318225 18330 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.280772 18266 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.319957 18330 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.320199 18266 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:47.282884 18243 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.321182 18243 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.322376 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:47.322999 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 198926us
I20260709 13:03:47.323544 18065 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:47.318799 18311 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:47.325065 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:47.325181 18065 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:47.325783 15847 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:47.326426 15158 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:47.327008 18065 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:47.327668 18065 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.329001 18265 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:47.323212 18361 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 13:03:47.329536 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 107973us
I20260709 13:03:47.330628 18418 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.333019 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:47.294872 18188 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.335244 18381 raft_consensus.cc:1064] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:03:47.275624 18215 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.337131 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:47.325573 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.339792 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.327770 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:47.333133 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.349485 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.352983 15159 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:47.348953 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.340562 18332 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:03:47.360497 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.361096 18332 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:47.361192 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:47.361631 15159 raft_consensus.cc:2410] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:47.361706 18332 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.362895 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.363134 18332 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.364142 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.367945 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:47.354416 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.378504 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.308679 18321 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:03:47.386158 15839 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:47.369714 15159 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:47.371994 18332 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:47.355021 17831 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:47.302748 18227 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:03:47.349370 18131 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:47.294483 18410 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:47.430084 18215 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.379909 18065 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.433197 18245 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:47.442874 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81825.5us
I20260709 13:03:47.443917 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:47.400049 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.400202 18359 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:47.400967 18374 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.758s	user 0.007s	sys 0.011s
W20260709 13:03:47.401341 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.401574 18339 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:47.401928 15160 raft_consensus.cc:2410] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:03:47.401329 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 862) took 3310 ms. Trace:
I20260709 13:03:47.294013 18378 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:03:47.418880 18044 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.408857 17831 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.422453 18381 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:47.424482 18356 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.695s	user 0.017s	sys 0.000s
I20260709 13:03:47.424996 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 1968) took 2153 ms. Trace:
I20260709 13:03:47.397604 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.390789 18265 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:47.456967 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 183344us
W20260709 13:03:47.457653 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57765.4us
I20260709 13:03:47.430152 18410 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:47.466079 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 143761us
W20260709 13:03:47.477154 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 109372us
I20260709 13:03:47.446316 18311 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:47.281127 18327 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:47.486020 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.487172 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 824) took 1117 ms. Trace:
I20260709 13:03:47.462622 15844 rpcz_store.cc:276] 0709 13:03:44.091476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.074190 (+2982714us) service_pool.cc:224] Handling call
0709 13:03:47.156658 (+ 82468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.157778 (+  1120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.157819 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.158417 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.244402 (+ 85985us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:47.244461 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.244563 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:47.401112 (+156549us) raft_consensus.cc:1685] finished
0709 13:03:47.401130 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.401252 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":167,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":85461,"threads_started":1,"wal-append.queue_time_us":240228}
I20260709 13:03:47.461750 18339 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.498965 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 879) took 4013 ms. Trace:
I20260709 13:03:47.501310 18265 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:47.505060 18359 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:47.507294 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.509382 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.510285 18245 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:47.479743 15162 rpcz_store.cc:276] 0709 13:03:45.191876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.870028 (+1678152us) service_pool.cc:224] Handling call
0709 13:03:46.870586 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.871194 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:47.065325 (+194131us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.341823 (+276498us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:47.342196 (+   373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:47.342355 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.342494 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.344577 (+  2083us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321760,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":136068,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":292366,"threads_started":2,"wal-append.queue_time_us":342864}
I20260709 13:03:47.511451 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.521727 18245 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:47.512408 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.494941 15154 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:47.523484 15087 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:47.523592 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.513283 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:47.513878 15079 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:47.524461 18245 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:47.478322 18374 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.478561 17831 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.515250 15083 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:47.461016 18235 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.525676 18361 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:47.475332 18264 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:47.496486 15082 rpcz_store.cc:276] 0709 13:03:46.369331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.586508 (+217177us) service_pool.cc:224] Handling call
0709 13:03:46.775213 (+188705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.281626 (+506413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.486860 (+205234us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540011a3d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.487100 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118050048}
I20260709 13:03:47.443899 18243 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.518180 15846 rpcz_store.cc:276] 0709 13:03:43.485757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.627977 (+3142220us) service_pool.cc:224] Handling call
0709 13:03:46.628238 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.865604 (+237366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.489280 (+623676us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.498745 (+  9465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100343040}
I20260709 13:03:47.495110 18311 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.533281 15843 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 7. Ops: []
W20260709 13:03:47.536278 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 356262us
I20260709 13:03:47.532629 15233 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:47.541656 15087 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:03:47.537003 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 184002us
I20260709 13:03:47.519163 18339 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.566457 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.567852 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:47.559675 18311 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:47.573798 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44433.3us
I20260709 13:03:47.574867 18243 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.575688 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.575939 18378 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:47.560971 18264 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:47.579371 18374 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:47.581203 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.579382 18265 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.581866 18264 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.583133 18341 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:47.583663 18341 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.584095 18341 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:47.581892 18374 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:03:47.586943 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56637.2us
I20260709 13:03:47.535037 18361 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.590791 18361 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:47.593807 18235 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.596005 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.596686 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:47.597620 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 100406us
I20260709 13:03:47.596701 15085 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.599107 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.597203 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.600620 15087 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:47.602972 15083 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:47.607775 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.608505 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.616341 15088 raft_consensus.cc:2368] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
W20260709 13:03:47.617408 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.618129 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.618634 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.623124 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.623910 15079 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:47.624380 18437 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:47.625319 18437 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:47.625783 18437 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:47.626785 18437 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.629215 18434 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:03:47.632022 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 186012us
I20260709 13:03:47.635854 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.637187 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.639503 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.644577 15082 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:47.651165 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.652025 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:47.666771 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.670779 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.671253 15083 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:47.671129 18439 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:03:47.675961 18156 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.679388 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 863) took 3577 ms. Trace:
I20260709 13:03:47.679803 15843 rpcz_store.cc:276] 0709 13:03:44.102165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.084468 (+2982303us) service_pool.cc:224] Handling call
0709 13:03:47.210893 (+126425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.560264 (+349371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.679131 (+118867us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54004a7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:47.679310 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736830848}
I20260709 13:03:47.685197 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:47.686775 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.687076 15236 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:47.687533 15088 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:47.686767 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.689247 15235 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8.
I20260709 13:03:47.670866 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:47.686703 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 259762us
I20260709 13:03:47.677883 18156 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:47.691565 15202 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 3.363s (SENT) [suppressed 42 similar messages]
W20260709 13:03:47.694859 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.694881 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.696107 18308 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:03:47.696492 15083 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:47.699117 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:47.700986 15229 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.702574 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:47.641940 17914 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:03:47.704685 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.705250 15234 raft_consensus.cc:2368] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:47.728492 15085 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:47.737226 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.738133 15083 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:47.734115 18435 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:03:47.734616 18156 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:47.744738 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 176645us
W20260709 13:03:47.769898 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.782769 15236 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:47.783800 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.786110 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.787086 15085 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8. Re-sending same reply.
I20260709 13:03:47.788489 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.794307 15235 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:47.751224 18156 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:47.797729 18156 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.799214 18156 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.799914 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.800354 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.801223 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.802109 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.803854 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.752791 18361 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.805843 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 864) took 3701 ms. Trace:
I20260709 13:03:47.806231 15845 rpcz_store.cc:276] 0709 13:03:44.104748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.091144 (+2986396us) service_pool.cc:224] Handling call
0709 13:03:47.249742 (+158598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.319612 (+ 69870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.319645 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.320205 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.447428 (+127223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:47.447481 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.447571 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:47.805644 (+358073us) raft_consensus.cc:1685] finished
0709 13:03:47.805665 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.805791 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5727,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":126799,"threads_started":1,"wal-append.queue_time_us":281198}
I20260709 13:03:47.807610 18361 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:47.808238 15087 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:47.808314 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.808727 15087 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:47.809270 15079 raft_consensus.cc:2368] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:47.809646 15087 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:47.798105 18370 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:47.810708 18370 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:47.813208 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.813416 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.814285 15088 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:47.810671 15087 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:47.815677 18427 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:03:47.796670 18131 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:47.817304 18131 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:47.817731 15080 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:47.818586 18131 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.820117 18131 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.820968 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.812364 18370 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.822607 18370 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:47.832978 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:47.834913 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.853678 15846 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:47.833186 14984 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 3 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:47.857435 15087 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:47.833003 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52398.4us
W20260709 13:03:47.864548 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.828605 18265 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:47.895825 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:47.896755 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:47.796423 18311 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.902612 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:47.905761 15839 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:47.907174 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:47.907519 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.914899 15083 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:47.903056 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:47.916625 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:47.917081 15163 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:47.921554 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.935225 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.936028 15155 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:47.936326 15233 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:47.937999 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:47.833796 18339 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:47.899345 18265 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.908946 14984 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:48.002961 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 325695us
I20260709 13:03:48.091317 15163 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:48.019307 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.094059 15231 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.060958 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:48.095204 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.026117 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:48.096719 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.097280 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 405714us
W20260709 13:03:48.084740 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 516939us
W20260709 13:03:48.003382 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100455us
W20260709 13:03:48.098850 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.105691 14982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 320) took 27008 ms (client timeout 15000 ms). Trace:
I20260709 13:03:48.106739 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.107400 18311 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:48.109014 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.113605 18438 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:03:48.115525 15236 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:48.116725 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.965608 18245 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.002314 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:48.088562 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 389536us
W20260709 13:03:47.944113 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 294534us
I20260709 13:03:47.984220 15155 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:47.965780 15162 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:47.972609 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:47.945793 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.019140 15235 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:48.117483 15236 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:47.941980 18265 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.129935 15154 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:48.117444 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.134310 15228 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:48.138078 15233 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:03:48.144111 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.108759 18447 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:03:48.145790 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:47.984702 14984 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 3 to 6, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 6 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:48.155229 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.166380 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.166934 15157 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.166013 15083 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:48.175669 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:48.176749 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 193073us
I20260709 13:03:48.177345 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.178987 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 358210us
I20260709 13:03:48.156394 18264 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:48.186507 15158 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8. Re-sending same reply.
I20260709 13:03:48.187937 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.165120 17880 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.178591 18381 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:48.192601 14984 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 3 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.207602 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.151592 18065 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:03:48.178512 18339 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:48.178251 18311 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:48.209967 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.186409 18264 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.218716 15234 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:48.207132 17831 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.006s
I20260709 13:03:48.216198 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:48.212081 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.221019 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 75298.5us
I20260709 13:03:48.220995 18264 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.227703 18339 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:03:48.229411 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 99880.1us
I20260709 13:03:48.230535 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:03:48.233176 18445 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:03:48.235282 17831 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:48.237447 18467 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:48.238142 17831 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:48.238914 15082 raft_consensus.cc:2368] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:48.240129 17880 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.242544 17880 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:48.243569 18440 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:48.243961 18440 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:48.244809 18440 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.246125 18440 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.246253 15230 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:48.221141 18369 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:48.247380 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.247983 15230 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.247771 18369 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.250499 18369 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:48.236686 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:48.218058 18266 consensus_meta.cc:287] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.897s	user 0.009s	sys 0.004s
I20260709 13:03:48.226965 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.252302 15088 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:48.258486 15157 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
W20260709 13:03:48.257371 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.257742 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50840.6us
I20260709 13:03:48.261503 15158 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:48.246127 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:48.181216 17914 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:03:48.268740 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.269770 18439 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:48.270253 18439 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.270632 18439 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.270509 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.271654 15232 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
W20260709 13:03:48.106207 14982 rpcz_store.cc:269] 0709 13:03:21.002926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.047152 (+ 44226us) service_pool.cc:224] Handling call
0709 13:03:48.002695 (+26955543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:38.149190 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:38.150023 (+   833us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:38.150093 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:03:38.150113 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:38.151494 (+  1381us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:38.151593 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:38.151637 (+    44us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:03:38.152779 (+  1142us) tablet.cc:701] Row locks acquired
0709 13:03:38.183219 (+ 30440us) write_op.cc:260] PREPARE: finished
0709 13:03:38.256238 (+ 73019us) write_op.cc:270] Start()
0709 13:03:38.308720 (+ 52482us) write_op.cc:276] Timestamp: P: 1783602218225078 usec, L: 0
0709 13:03:38.308744 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:03:38.309548 (+   804us) log.cc:844] Serialized 4084 byte log entry
0709 13:03:39.811530 (+1501982us) op_driver.cc:464] REPLICATION: finished
0709 13:03:39.904768 (+ 93238us) write_op.cc:301] APPLY: starting
0709 13:03:39.904911 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:39.905162 (+   251us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:39.905294 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:39.906830 (+  1536us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:40.005783 (+ 98953us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:03:40.009506 (+  3723us) write_op.cc:312] APPLY: finished
0709 13:03:40.093104 (+ 83598us) log.cc:844] Serialized 77 byte log entry
0709 13:03:40.128578 (+ 35474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:40.143391 (+ 14813us) write_op.cc:454] Released schema lock
0709 13:03:40.160818 (+ 17427us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:40.195995 (+ 35177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:40.198352 (+  2357us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8445442,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":92901,"apply.run_cpu_time_us":5977,"apply.run_wall_time_us":267241,"num_ops":10,"prepare.queue_time_us":466506,"prepare.run_cpu_time_us":17626,"prepare.run_wall_time_us":413180,"raft.queue_time_us":667622,"raft.run_cpu_time_us":7277,"raft.run_wall_time_us":116986,"replication_time_us":1502561,"thread_start_us":1036002,"threads_started":4,"wal-append.queue_time_us":296452,"wal-append.run_cpu_time_us":22978,"wal-append.run_wall_time_us":2554254}]]}
I20260709 13:03:48.273590 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.274261 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.275187 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.275900 15155 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:48.276077 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.271027 18462 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:48.276329 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.279546 15162 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:48.266239 14984 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:48.279981 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.283615 15234 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:03:48.260769 18341 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.005s
I20260709 13:03:48.217013 18434 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:48.289603 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.290434 15234 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:48.290916 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.293236 18341 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:48.293951 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.294426 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.295089 18378 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:48.295482 18378 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:48.295580 15163 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:03:48.296732 15234 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:48.296463 18378 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.297820 18378 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.297878 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.299074 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.299801 15234 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:48.299835 18461 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:03:48.300630 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.300913 15843 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:48.301726 18434 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:48.266399 18458 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:48.302158 18434 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:48.303995 15202 leader_election.cc:400] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:48.304294 15202 leader_election.cc:403] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.304503 18434 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.305039 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.305760 18434 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.306465 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:48.307324 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:48.308187 18327 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:48.308539 18327 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.260736 18311 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.309237 18327 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.309336 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.310156 15231 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:48.310845 18311 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.302119 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.313366 18361 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:48.313848 18361 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.314227 18361 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.315521 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.265008 18266 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.319438 18266 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:48.320344 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.321159 15228 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:48.322057 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.322762 15845 raft_consensus.cc:2376] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5. Re-sending same reply.
I20260709 13:03:48.324285 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:48.325196 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:48.325557 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.326648 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:48.326741 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.327555 15840 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.327962 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.328737 15156 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:48.329025 18339 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.329201 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.329916 18462 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:48.330430 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47359.6us
I20260709 13:03:48.329227 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:48.333606 15841 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:48.333477 18265 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.333768 15160 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:48.334471 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 110869us
W20260709 13:03:48.340981 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 163680us
I20260709 13:03:48.341511 15843 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:48.336457 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:48.346168 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.346984 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:48.345297 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.351390 18265 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:48.352684 15229 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.004s
W20260709 13:03:48.352692 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 106641us
I20260709 13:03:48.353307 15229 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:48.353796 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:48.356051 15839 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:48.358893 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:48.362077 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53449.4us
I20260709 13:03:48.361783 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.336402 18341 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.375317 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.375990 15839 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:48.363428 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.364912 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:48.354452 17880 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:48.405818 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48092.4us
W20260709 13:03:48.406903 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 60863.1us
I20260709 13:03:48.407606 15844 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:48.381723 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.409942 15845 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.361793 18266 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.411062 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57408.6us
I20260709 13:03:48.364522 17914 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:48.414228 15810 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:48.418008 15203 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.519s (SENT)
I20260709 13:03:48.410007 14984 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.362581 18265 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.418708 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.381599 18369 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.428687 18264 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.429315 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.432540 14984 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.331175 18462 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.436755 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.470744 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:48.437157 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 101850us
I20260709 13:03:48.438441 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:48.438314 18452 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:48.375140 18434 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:03:48.442391 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.443132 15156 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:48.444197 15162 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:48.405371 17880 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.445349 15155 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:03:48.445020 18266 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.445536 15158 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:48.382176 18339 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.490183 14984 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.461107 15846 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:48.453447 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.466485 15230 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:48.467157 15847 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:48.475189 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.482537 15160 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:48.482908 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.455415 18264 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.505818 14984 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:48.450762 18156 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
I20260709 13:03:48.508688 15051 leader_election.cc:400] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8.
I20260709 13:03:48.508167 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:48.512379 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74017.6us
I20260709 13:03:48.488255 17880 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.486907 18311 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.514098 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.518105 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96863.3us
I20260709 13:03:48.468886 15839 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 4.
I20260709 13:03:48.496822 18131 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:48.497339 18418 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:48.522987 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.525105 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 119673us
I20260709 13:03:48.507979 18458 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:48.530441 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46786.8us
W20260709 13:03:48.535822 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 121650us
W20260709 13:03:48.538236 15203 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.641s (SENT)
I20260709 13:03:48.535336 15846 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:48.448827 18327 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.534430 18156 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:48.551599 18156 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:48.553095 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.498795 18361 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.518711 18265 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:48.568012 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:48.569859 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.569926 15228 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:48.570549 15156 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.572609 15158 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:48.570019 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.528375 18475 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:48.575042 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 102812us
W20260709 13:03:48.575290 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 153392us
I20260709 13:03:48.573067 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:48.575971 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:48.578490 15163 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.564936 14984 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 3, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 3 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.555050 18374 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:48.580335 15236 raft_consensus.cc:2368] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:48.585795 15051 leader_election.cc:400] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:48.586689 15051 leader_election.cc:403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.587904 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.588806 15230 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.570183 18330 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:48.591878 18361 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.580418 18475 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.570055 18265 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:48.595923 18475 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.596220 18265 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.596997 18266 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:48.569314 18454 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:48.599098 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 62391.8us
I20260709 13:03:48.605477 14984 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 6 to 7, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 7 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.607571 15235 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:48.530068 18311 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.611765 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.613010 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:48.614909 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51434.5us
I20260709 13:03:48.614944 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.555248 18434 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:03:48.563392 18369 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:03:48.624892 15156 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:48.595916 18321 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:48.621058 14984 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 4, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:03:48.632170 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45222.6us
I20260709 13:03:48.609215 18378 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:48.637212 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67553.1us
I20260709 13:03:48.636382 15234 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:48.637840 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.636523 18044 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:48.636814 18469 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:48.646873 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.638548 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.641714 15230 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:48.634824 15126 connection.cc:256] RPC call timeout handler was delayed by -1.6386318850001089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:03:48.644830 15839 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:48.640547 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:48.653784 15126 connection.cc:256] RPC call timeout handler was delayed by -1.5810318850000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:03:48.654351 15126 connection.cc:256] RPC call timeout handler was delayed by -1.5684318850001091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260709 13:03:48.654798 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3983318850000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 13:03:48.655220 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3803318850000323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 13:03:48.655646 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3452318850000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 13:03:48.656059 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3038318850000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260709 13:03:48.633101 18065 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:48.659230 14984 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.631310 18215 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6.
I20260709 13:03:48.657927 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.586099 18156 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.660951 15231 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.643577 18044 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:48.663636 18044 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.646221 18469 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:48.661480 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.631201 18321 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.659703 15848 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
W20260709 13:03:48.659042 15126 connection.cc:256] RPC call timeout handler was delayed by -1.2327318850000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:03:48.666042 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:48.661621 18474 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:48.637413 18311 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:48.670239 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 60957.4us
I20260709 13:03:48.674562 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:48.634588 17914 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:03:48.686079 18065 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:48.686677 18065 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.668536 18044 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:48.689116 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73326.5us
I20260709 13:03:48.689749 17831 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:48.689971 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.690644 17831 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:48.691095 17831 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:48.691291 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.692024 17831 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.679216 18374 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:48.697824 18327 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.697741 18374 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.680233 18457 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:48.681540 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:48.705024 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:48.682530 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 82730us
I20260709 13:03:48.682855 18340 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:48.683135 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83017.3us
I20260709 13:03:48.683641 15844 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:48.706977 18473 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:48.707594 18266 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:48.707805 18473 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.708372 18473 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.683748 18469 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.709854 18469 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.689026 18381 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:48.707001 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.693005 18245 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:48.694357 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:48.694808 18455 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:48.694993 18330 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:48.695698 17914 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:48.716686 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:48.678741 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71523.9us
I20260709 13:03:48.750706 15230 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:48.672781 18321 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.688694 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:48.673328 14984 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.707257 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:48.683673 18339 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:48.659950 18369 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:48.675603 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.720116 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.716560 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.915097 15235 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:48.748881 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 72825.1us
I20260709 13:03:48.750279 18327 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:48.783331 15846 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:48.917542 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:48.733702 17914 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:48.916674 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 248597us
I20260709 13:03:48.744762 18215 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.732892 18340 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:48.786741 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79528.8us
I20260709 13:03:48.714942 18455 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:48.802874 18439 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.531s	user 0.014s	sys 0.025s
I20260709 13:03:48.822086 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.940572 15230 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 4: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.821898 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.753558 18457 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.824193 18418 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:48.825881 18381 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:48.840037 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 148835us
I20260709 13:03:48.943876 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.857081 15236 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:48.946743 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:48.857415 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 163118us
W20260709 13:03:48.859496 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 170855us
I20260709 13:03:48.813259 18374 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:48.783130 18339 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:48.862901 14984 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.866083 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:48.951790 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 85789.8us
I20260709 13:03:48.870432 15237 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:48.901098 18361 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.905962 18265 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.906255 18473 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.922888 15163 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:48.899986 17880 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.958272 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:48.958284 14984 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.961602 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.928886 15847 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:48.943099 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:48.832579 18065 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:48.931291 18369 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:48.944099 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:48.940816 18044 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:03:48.789891 17831 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:48.951264 18457 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.977110 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.986043 15231 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:48.985795 18475 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.977869 18131 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:48.987468 15160 raft_consensus.cc:2368] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:48.942281 18439 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.988201 18475 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:48.948151 15202 leader_election.cc:400] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:48.953504 18339 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.988997 15202 leader_election.cc:403] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.989753 18439 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:48.962976 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 141190us
I20260709 13:03:48.990703 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:48.990953 18440 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.744s	user 0.012s	sys 0.000s
I20260709 13:03:48.964710 14984 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 5, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.992295 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:48.945807 18381 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:48.993074 18381 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.992394 14984 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 4, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 4 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:48.969079 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.969825 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.995882 15233 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:48.970221 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.968500 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:48.971581 15236 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:48.972317 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:48.997781 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.998610 15839 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:03:48.982084 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64652.7us
I20260709 13:03:48.956348 18374 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:48.951488 15846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:03:48.973004 17914 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.962677 18361 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:48.956401 18327 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.962723 18215 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:48.956161 18311 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:48.954818 18497 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.975751 15229 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:48.986939 18457 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:48.974131 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:49.012248 15839 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:49.065528 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:49.121749 15157 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:49.122090 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.122665 15848 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:49.122905 15839 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:49.056861 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 112811us
I20260709 13:03:49.062709 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.123697 15842 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:49.062592 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.000499 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:48.979579 15843 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:49.004873 15235 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:49.125169 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:48.979290 17880 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:48.995592 18227 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:49.001528 15846 raft_consensus.cc:606] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.155077 15846 raft_consensus.cc:614] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:48.966300 18454 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:49.117105 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.160135 15841 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.142266 15233 raft_consensus.cc:2410] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:49.144037 15844 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 13:03:49.067181 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:48.943897 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 196546us
I20260709 13:03:48.973243 18369 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.122169 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.171989 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:49.045835 18065 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:48.981040 18473 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.125214 18418 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:49.122876 18330 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:03:49.136684 18311 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:49.136898 18440 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.145594 18266 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:49.138058 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.138664 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 146452us
I20260709 13:03:49.139828 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:49.147827 18496 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.242s	user 0.000s	sys 0.003s
I20260709 13:03:49.004393 18131 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:49.153097 18497 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:49.200107 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.200582 18245 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:49.201115 18245 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:49.202145 18497 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.203455 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.205155 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:48.982504 18156 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:49.065668 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 75678.1us
I20260709 13:03:49.158890 15237 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.004110 18495 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:49.174005 15156 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:49.177762 18369 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:49.172693 15163 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:49.175884 15845 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:49.176121 15157 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:49.170635 15839 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:49.169797 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 195710us
I20260709 13:03:48.983248 18498 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.142627 18321 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:49.179248 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.179554 18044 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:49.231747 18044 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.232378 18340 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.179634 18361 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:49.181113 18499 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:03:49.233714 18466 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:49.234249 18466 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.234630 18466 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.181748 15155 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:49.236956 18044 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.175011 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.237928 15848 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 4: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:49.239202 15848 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:49.239735 18321 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.181787 15846 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:49.237959 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.242095 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.242106 18458 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:49.243081 18458 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.243501 18458 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.245054 18341 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:49.249758 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52829.8us
I20260709 13:03:49.251367 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.187134 15841 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:49.196045 18454 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:49.192902 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.207381 18439 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:49.194931 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 204318us
I20260709 13:03:49.262491 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.143457 17914 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.184108 18327 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:49.202056 18245 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:49.189834 18418 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.174667 15842 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:49.222600 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82848.1us
I20260709 13:03:49.230247 18497 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:49.221876 18266 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:49.254601 15202 leader_election.cc:400] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:49.255883 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:49.240556 15848 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:49.177030 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.264384 18327 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.156244 18227 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.213313 18495 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.271821 15841 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:49.274688 15814 leader_election.cc:400] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:49.293910 15814 leader_election.cc:403] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:49.144375 18506 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.063233 18455 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.275535 15846 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:03:49.281946 15202 leader_election.cc:403] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:49.193813 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 149672us
I20260709 13:03:49.261551 18439 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.208431 18440 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.313027 18439 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.314669 18440 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.226225 18330 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:03:49.226012 18264 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:03:49.272614 18245 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.316780 18264 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:49.317816 18264 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:49.276644 18418 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.330595 18509 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:03:49.331178 18265 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
W20260709 13:03:49.289728 18501 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:49.290133 18503 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:03:49.273767 18457 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.293391 18227 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.295969 15842 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:49.351667 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.298023 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.257591 18131 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:03:49.299937 18266 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.300822 18308 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:49.354707 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.287771 18327 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:49.301502 18454 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.285388 18156 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:49.282866 18498 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.304459 15088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:49.368098 15088 raft_consensus.cc:606] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:49.300769 18492 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.295s	user 0.001s	sys 0.001s
I20260709 13:03:49.259325 18381 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:49.308975 18311 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.278942 18497 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.310652 18458 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.321897 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:49.295333 18361 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:03:49.318688 18264 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.295482 18495 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.334587 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:49.383389 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 132119us
I20260709 13:03:49.335737 15157 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:49.356787 18503 consensus_queue.cc:1243] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:49.343369 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:49.295738 15237 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:49.348852 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:49.390237 15845 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:49.350064 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 87652.5us
I20260709 13:03:49.213961 18374 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.355120 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.301024 18502 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.374845 18440 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:49.364015 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:49.398097 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.346743 18506 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.346933 18341 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.346530 18330 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6.
I20260709 13:03:49.378885 18156 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:49.376951 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79013.3us
W20260709 13:03:49.378027 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.379809 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58030.3us
I20260709 13:03:49.378810 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.342686 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:49.384285 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:49.383665 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49171.3us
W20260709 13:03:49.379910 18469 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.003s
I20260709 13:03:49.393079 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:49.361779 15086 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:49.422078 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.379238 18497 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.399467 15848 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:49.403623 15079 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:49.405103 15034 auto_rebalancer.cc:629] Failed to schedule move for tablet ff3342a5b2ad49e2b252d7cd47b9851d: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:49.393851 18473 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:49.433435 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.434428 15082 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:49.434482 15085 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:49.398605 18503 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:49.427814 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43625.3us
I20260709 13:03:49.428175 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:49.401464 18308 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.421609 15840 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:49.423385 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:49.433660 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:49.443133 15083 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:49.440174 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.441524 15086 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:49.450134 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:49.450932 15154 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:49.447798 15086 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:49.454180 18065 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:49.454641 18065 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:49.455564 18065 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.410715 18361 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:03:49.451264 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.461494 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.450065 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.465348 15082 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:49.467235 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55492.8us
W20260709 13:03:49.467159 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59546.4us
I20260709 13:03:49.435405 18513 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.438454 18473 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:49.478286 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 250156us
I20260709 13:03:49.449011 18475 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.449302 18498 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:49.484197 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:49.427662 18434 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:03:49.477789 18340 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.438433 18339 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.517854 15847 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:49.519048 18339 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:49.519188 15082 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:49.520993 18516 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.462651 18509 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:03:49.481587 18065 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.481822 18264 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.472590 18503 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:49.500061 18361 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.512945 18471 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:49.468746 18044 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:03:49.429705 18466 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.556464 18361 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.557129 18361 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.558951 18466 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.559315 18361 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.434163 18502 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:49.526361 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.463579 17831 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:49.516260 18245 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:49.565943 18518 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:03:49.567027 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.567377 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.569209 18339 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:49.449415 18308 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.437335 18506 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.574362 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:49.587427 15237 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:49.571897 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 623762us
W20260709 13:03:49.577900 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 109602us
I20260709 13:03:49.578032 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:49.494076 18498 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:49.560158 18418 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:49.596371 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.597654 18418 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.598055 18502 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:49.585929 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.481555 18520 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:03:49.570266 18339 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:03:49.493420 18465 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 13:03:49.585553 18340 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:49.601130 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:49.602133 15236 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:49.541929 18330 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:49.586062 18156 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.603982 18330 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:49.605011 18473 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.594069 15228 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:49.594666 18308 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.594758 18498 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:49.595443 18525 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.598994 18502 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:49.612214 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:49.618367 15229 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:49.553524 18471 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.625617 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.626646 15842 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:49.553822 18381 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.619534 15080 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.620199 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:49.618841 18517 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:49.621910 15843 raft_consensus.cc:1217] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-5.6]   Dedup: 2.5->[5.6-5.6]
I20260709 13:03:49.622125 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.635411 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:49.631492 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:49.637101 15082 raft_consensus.cc:606] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.637455 15082 raft_consensus.cc:614] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.632650 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:49.639475 15843 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 7. Ops: [2.5-5.6]
I20260709 13:03:49.640147 15228 raft_consensus.cc:2410] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:49.604588 18439 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.632503 18471 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:49.619860 18502 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:49.659375 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.635157 18458 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.663151 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:49.641517 18509 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:49.646590 18512 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:49.649024 18517 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:49.706184 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:49.707598 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.709676 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 226309us
W20260709 13:03:49.712713 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 265619us
I20260709 13:03:49.714550 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:49.672740 18462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.006s	sys 0.013s
I20260709 13:03:49.667863 18509 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:49.654412 18466 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:49.658725 18381 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:03:49.728351 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43790us
I20260709 13:03:49.683658 18215 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:49.643039 18330 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.663582 18520 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:03:49.734040 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:49.731462 18462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.006s	sys 0.014s
W20260709 13:03:49.739614 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 153803us
I20260709 13:03:49.742388 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.743738 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2529) took 1469 ms. Trace:
I20260709 13:03:49.744153 15161 rpcz_store.cc:276] 0709 13:03:48.273881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.274002 (+   121us) service_pool.cc:224] Handling call
0709 13:03:49.743568 (+1469566us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:49.743688 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3374829440}
I20260709 13:03:49.747452 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:49.747865 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:49.748252 15082 raft_consensus.cc:606] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.710103 18525 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:49.752657 15154 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:49.748757 15082 raft_consensus.cc:614] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.753510 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:49.730870 18452 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.758693 15159 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:49.759467 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:49.733201 18509 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:49.733134 18512 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:49.766450 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:49.738308 18374 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.750631 18266 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:49.771734 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:49.754519 18520 consensus_queue.cc:1243] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:49.755632 18265 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:49.733875 18458 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:03:49.788872 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.788208 18506 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.791237 18378 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:49.761068 18502 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.763612 18462 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.763970 18440 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.763813 18434 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.794698 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:49.769745 18512 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:49.776477 18311 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:49.794070 18524 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:03:49.816391 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:49.809231 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.821027 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54707.2us
I20260709 13:03:49.817337 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:49.822131 15082 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:49.795171 18378 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:49.816457 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:49.823163 18532 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:03:49.823429 18462 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:49.823701 18502 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:49.802268 18521 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:03:49.725044 18503 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.831980 18311 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:49.844009 18311 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.845186 18439 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:49.845983 18541 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:49.846828 18541 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:49.838533 18526 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:03:49.762049 18525 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:49.817109 18506 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:49.830910 18369 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.001s
I20260709 13:03:49.786288 18327 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.824085 18523 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.831373 18378 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:49.799217 18454 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:03:49.862268 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.787534 18452 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.875869 15080 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:49.829281 18434 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.829521 18440 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:49.883724 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79437.7us
I20260709 13:03:49.869073 18532 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:49.909507 15126 leader_election.cc:400] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:49.909950 15126 leader_election.cc:403] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:49.795527 18517 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.864935 18369 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:49.865376 18506 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:49.911813 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 155249us
I20260709 13:03:49.914575 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:49.919690 15085 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:03:49.911444 18495 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.007s
I20260709 13:03:49.906262 18532 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.910290 18541 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:49.851639 18512 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:49.922111 18522 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:03:49.931981 18341 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:49.932803 18506 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:49.925741 18529 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:49.931633 18531 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:03:49.943984 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:49.921878 18538 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:49.922619 18439 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.002666 15082 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:49.939005 18532 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:50.003659 18475 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:50.046746 18536 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:03:50.047746 18439 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:50.048002 18131 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:50.048489 18131 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:50.049520 18131 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.050585 18131 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:50.053202 18534 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:03:50.055009 18545 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:03:49.949836 18462 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:03:50.078590 18340 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:50.088218 15159 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:03:50.085599 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:49.942431 18495 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.091866 15845 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:50.019385 18529 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:03:50.142191 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.159461 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.163084 15843 raft_consensus.cc:1217] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-5.6]   Dedup: 2.5->[5.6-5.6]
I20260709 13:03:50.165714 15843 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 7. Ops: [2.5-5.6]
W20260709 13:03:50.165242 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 364049us
I20260709 13:03:50.048553 18546 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:03:50.170797 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.171797 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:50.176046 15157 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:50.172011 18475 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.172574 18434 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.087580 18532 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.176808 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.098932 18156 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:50.162223 18418 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.183710 15237 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:50.115984 18065 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.184764 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 240856us
I20260709 13:03:50.188146 15846 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:50.188637 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.190716 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.158427 18520 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.180682 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:50.191342 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 462997us
I20260709 13:03:50.191985 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:50.190965 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:50.192376 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.204905 15236 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:50.197819 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:50.199375 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.206677 15230 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:50.200373 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:50.207430 15161 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:50.186929 18044 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:50.158509 18361 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:03:50.214097 18044 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:50.222782 18361 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.187423 18327 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.098373 18340 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.212782 15228 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:50.212996 15845 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:50.219946 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.228739 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.229785 15085 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:50.228790 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49037.7us
W20260709 13:03:50.232064 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.226073 15846 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:50.239503 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:50.242184 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.223510 18361 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.209750 18506 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:50.248183 15085 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 5. Current term is 6.
I20260709 13:03:50.205606 18452 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.260277 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.257207 15080 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:50.261374 15085 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 5: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:50.248605 15083 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:50.186122 18475 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.217339 18525 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.234965 18469 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:50.247367 18374 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:50.271725 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.279186 18560 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.280602 18548 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.281250 18549 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:50.281683 18433 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:03:50.279397 18543 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:50.283634 18550 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:03:50.283874 18543 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:50.279207 18266 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:50.201174 18418 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:50.285972 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.210896 18434 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:03:50.250795 18473 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:50.222915 18044 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.294489 18340 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.294804 18475 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.206084 18065 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.287603 18440 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.301699 18523 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.300424 15085 pending_rounds.cc:85] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:03:50.297820 18374 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:03:50.316015 18227 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.945s	user 0.014s	sys 0.000s
I20260709 13:03:50.303188 18546 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:50.346972 18546 consensus_queue.cc:1243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:50.296123 18516 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:50.316104 17914 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:03:50.347558 18516 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:50.347795 15080 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:50.258283 17880 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:50.313509 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 76477.5us
I20260709 13:03:50.274399 18469 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.251818 18369 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.320348 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:50.298404 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.303200 18473 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.335000 15228 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:50.260746 18506 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.337260 15085 pending_rounds.cc:107] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:03:50.311594 18266 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.341017 15083 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:50.360597 15085 raft_consensus.cc:2889] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:50.342110 18466 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.345305 18553 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:03:50.345048 18227 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.346164 18418 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.346371 18065 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.356547 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:50.362063 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:50.355281 18498 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:03:50.316658 18525 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:50.369442 15080 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:03:50.360893 18497 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.910s	user 0.010s	sys 0.003s
W20260709 13:03:50.390110 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.391144 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.338826 18330 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:03:50.357717 18469 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.399586 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.359426 18327 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:50.406934 15156 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:50.378887 15086 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.408509 15233 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:50.361984 18506 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.362152 18473 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.284759 18543 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.281131 18548 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.418627 15087 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:50.418840 18543 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.419013 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:50.258301 18529 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.410295 15845 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:50.420960 15845 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:50.281854 18549 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.280017 18560 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.216351 18520 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.419569 15158 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:50.434870 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.373620 18495 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:50.322002 18245 consensus_queue.cc:1175] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 5 status { last_received { term: 5 index: 5 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)" } } last_received_current_leader { term: 5 index: 5 } }
I20260709 13:03:50.366905 18266 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.349458 18523 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.370695 18466 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.441471 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.759064 15845 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:50.420626 18548 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.359702 17880 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.767174 17880 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:50.521526 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 129677us
I20260709 13:03:50.779425 15843 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.781579 18520 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.782986 18227 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.784204 18555 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.784811 18555 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.787142 18555 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.782950 18466 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.522444 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:50.608621 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 185494us
I20260709 13:03:50.747349 18330 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.789824 15080 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:50.547135 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 188149us
I20260709 13:03:50.625939 15157 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:50.789860 18330 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.790827 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.791570 15080 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:50.441179 15128 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.840s (SENT)
I20260709 13:03:50.792121 18330 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.792194 15128 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:50.528108 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:50.625002 15088 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:50.793114 15156 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:50.739604 18327 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.793954 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:50.794055 18327 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:50.794641 15088 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:50.792798 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:50.616473 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 185954us
I20260709 13:03:50.407260 18497 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.797006 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:50.797832 15080 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:50.798283 18497 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:50.689951 18554 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:03:50.801009 18266 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:50.748380 18502 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
I20260709 13:03:50.806031 18502 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:50.806520 18502 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.806926 18502 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:50.807035 15080 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.753408 18361 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.618192 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:50.682950 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.655840 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 683) took 1082 ms. Trace:
I20260709 13:03:50.610146 18532 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.409746 18465 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:50.652277 18549 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.529597 15086 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:50.569689 18374 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:50.825418 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.825709 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.630752 18378 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.753s	user 0.005s	sys 0.008s
I20260709 13:03:50.433202 18516 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.657857 18560 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.839250 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.839600 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:50.839828 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.840432 15234 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:50.406459 18369 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:50.840673 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.825821 15079 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:50.796317 18327 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:50.758025 18523 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:50.830233 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.847558 15845 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:50.819278 15233 rpcz_store.cc:276] 0709 13:03:49.573852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.575717 (+  1865us) service_pool.cc:224] Handling call
0709 13:03:49.576123 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.417006 (+840883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.655480 (+238474us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b540043d354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:50.655726 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913734912}
W20260709 13:03:50.652539 18558 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
W20260709 13:03:50.831095 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.832211 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:50.840044 15161 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.002s
I20260709 13:03:50.857631 15237 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:50.858527 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.841809 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.843317 15158 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:50.839598 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:50.848801 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:50.849134 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:50.850425 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:50.851213 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.861856 15232 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:50.851898 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.862676 15229 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:50.853966 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.399358 18498 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.855515 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:50.857506 15161 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:50.827473 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.865926 15237 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:50.836292 18361 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.867604 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 437664us
I20260709 13:03:50.868512 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.868825 15156 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
W20260709 13:03:50.846261 18341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.006s	sys 0.008s
I20260709 13:03:50.869669 15236 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:50.824223 18465 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:50.870477 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.873315 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:50.842269 18227 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.880223 15088 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:50.882838 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:50.881080 18341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.007s	sys 0.008s
I20260709 13:03:50.884137 15230 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:50.885334 18570 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 13:03:50.887362 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:50.892568 15237 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:50.894894 15084 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:50.898190 15087 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:50.857950 18555 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.901566 15845 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:50.903813 15080 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:50.869717 18369 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:50.842115 18378 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.907861 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:50.910375 18580 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:03:50.910913 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:50.911358 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:50.882829 18361 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.913089 18546 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:50.869545 18560 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:50.913168 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:50.882469 18523 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.844771 18567 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:50.898669 18520 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:50.930157 15229 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:50.898867 18497 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:50.940277 15080 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:03:50.913480 18530 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.182s	user 0.007s	sys 0.001s
I20260709 13:03:50.953060 15848 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:03:50.953794 18509 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.179s	user 0.006s	sys 0.007s
I20260709 13:03:50.893164 18227 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.963044 15841 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:50.955174 18506 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:03:50.967088 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:50.907552 18555 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:50.964902 18520 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.993342 15843 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:50.990332 18381 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:51.001024 18381 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:51.002245 18381 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.003367 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 896) took 1081 ms. Trace:
I20260709 13:03:50.980234 18321 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:50.954660 18330 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:03:51.005234 18321 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:51.005435 15203 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.106s (SENT)
I20260709 13:03:51.006186 18227 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.007337 18361 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:51.013104 18548 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:51.013967 18538 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:51.014380 18538 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:51.015202 18538 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.016405 18538 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:51.021281 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:51.022889 18567 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.023434 18567 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.003767 15840 rpcz_store.cc:276] 0709 13:03:49.922283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.922397 (+   114us) service_pool.cc:224] Handling call
0709 13:03:49.945814 (+ 23417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.954042 (+  8228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.954087 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.738053 (+783966us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b2c000fb978. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:50.738511 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.905248 (+166737us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:50.905284 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.905358 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.003075 (+ 97717us) raft_consensus.cc:1685] finished
0709 13:03:51.003094 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.003290 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1365,"raft.queue_time_us":67247,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4303,"spinlock_wait_cycles":11357952,"thread_start_us":166356,"threads_started":1,"wal-append.queue_time_us":262810}
I20260709 13:03:51.028581 15234 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:50.978073 18574 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.035863 15235 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:50.962476 18523 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.952009 18546 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.025548 18520 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.978981 18509 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.985742 18506 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:51.040413 18471 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.375s	user 0.010s	sys 0.004s
I20260709 13:03:51.006202 18321 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:50.965521 18465 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:51.055657 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 206579us
W20260709 13:03:51.074132 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 632951us
I20260709 13:03:51.056303 18559 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:51.056527 18555 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:51.075814 15128 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.987s (SENT)
I20260709 13:03:51.079046 15128 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.077170 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:51.079617 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.079916 15080 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:51.081636 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.086863 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:51.087600 15154 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.087713 15234 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:51.096109 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.076622 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.097911 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.102137 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.102273 15840 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:51.104024 15080 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:51.105654 15080 pending_rounds.cc:85] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:03:51.106065 15080 pending_rounds.cc:107] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:51.106346 15080 raft_consensus.cc:2889] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.108274 15085 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:51.123497 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.128311 18529 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:51.129913 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:51.072472 18434 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:51.078624 18559 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.134778 15842 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:51.137919 15085 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:51.143471 15088 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:51.032766 18369 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.145970 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:51.146908 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:51.045446 18574 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.131140 18465 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.129913 18555 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:51.170912 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 95132.3us
I20260709 13:03:51.133225 18498 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:51.168586 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.087930 18471 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.172204 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.131803 18567 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:51.190477 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:51.145128 18559 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.196677 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.149012 18528 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:51.129978 18588 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:51.137952 18529 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:51.192065 15085 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:51.212636 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.370344 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:51.359845 18537 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.445s	user 0.000s	sys 0.003s
W20260709 13:03:51.414297 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 242148us
W20260709 13:03:51.348434 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.147698 18502 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.585041 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 859) took 1790 ms. Trace:
I20260709 13:03:51.301060 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:51.729723 15084 rpcz_store.cc:276] 0709 13:03:49.794380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.794528 (+   148us) service_pool.cc:224] Handling call
0709 13:03:49.820589 (+ 26061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.895423 (+1074834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.584639 (+689216us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400118854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:51.584923 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470137728}
I20260709 13:03:51.351323 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2000) took 1557 ms. Trace:
W20260709 13:03:51.277370 18156 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.071s	user 0.019s	sys 0.000s
I20260709 13:03:51.731061 15159 rpcz_store.cc:276] 0709 13:03:49.793477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.793639 (+   162us) service_pool.cc:224] Handling call
0709 13:03:49.798932 (+  5293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.088757 (+289825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.350885 (+1262128us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540013e7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:51.351224 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664040832}
I20260709 13:03:51.605218 18458 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.277112 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.168063 18266 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:51.425963 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.210309 18434 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:51.337575 18568 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.729s	user 0.003s	sys 0.000s
I20260709 13:03:51.194658 18378 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.298005 18560 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.269474 18528 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.327052 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:51.319582 18529 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.315003 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.257251 15156 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:51.753646 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.277822 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.369005 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.320106 18523 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:51.363826 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.361707 18455 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.313032 18589 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.536301 15232 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.000s
I20260709 13:03:51.337890 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:51.761559 15232 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:51.762003 15842 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 6: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:51.363902 18227 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:03:51.483074 18588 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:51.609789 18374 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.646085 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 857) took 1851 ms (client timeout 2155 ms). Trace:
W20260709 13:03:51.602700 18473 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.002s
I20260709 13:03:51.364148 18330 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:51.781509 18330 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.782199 18330 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.659699 18340 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:51.783877 18340 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.786588 18340 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:51.786386 18588 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.329926 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:51.789721 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 459946us
I20260709 13:03:51.790406 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 939) took 2168 ms. Trace:
I20260709 13:03:51.790711 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.792788 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 675) took 1852 ms. Trace:
I20260709 13:03:51.380189 18546 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.794071 18502 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.794736 18502 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.795231 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.795770 18582 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:51.796041 15080 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:51.378430 18471 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.501346 18311 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:51.405386 18573 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.443s	user 0.003s	sys 0.000s
W20260709 13:03:51.545285 18543 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.126s	user 0.008s	sys 0.005s
I20260709 13:03:51.402349 18512 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.337934 18548 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:03:51.349817 18264 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:51.278683 15157 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:51.266204 15086 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:03:51.817859 15086 pending_rounds.cc:85] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:03:51.818173 15086 pending_rounds.cc:107] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:03:51.818567 15086 raft_consensus.cc:2889] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.346872 18598 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:51.232565 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.704248 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.724472 15155 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:51.372551 15845 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:51.728024 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.728637 18538 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.008s
I20260709 13:03:51.721827 18361 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.743170 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.747634 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:51.749069 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.091s	sys 0.208s
I20260709 13:03:51.749472 18580 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:51.829973 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.803177 18592 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:51.810469 18546 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:51.759992 15160 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:51.835184 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.738198 18156 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:51.839272 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.823889 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:51.755254 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.768854 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:51.751786 18517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.330s	user 0.008s	sys 0.007s
W20260709 13:03:51.618184 15154 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.003s
I20260709 13:03:51.841807 15154 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:51.753298 18560 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.795154 18502 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.297719 18381 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.400892 18044 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:51.790776 15839 rpcz_store.cc:276] 0709 13:03:49.621733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.622353 (+   620us) service_pool.cc:224] Handling call
0709 13:03:49.622691 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.623180 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:49.623445 (+   265us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.685889 (+ 62444us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.689172 (+  3283us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.705450 (+ 16278us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:49.705503 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.705574 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.359104 (+1653530us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:51.790078 (+430974us) raft_consensus.cc:1685] finished
0709 13:03:51.790181 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.790325 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15343,"prepare.queue_time_us":1370,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":1814,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":16681,"spinlock_wait_cycles":13239168,"thread_start_us":61188,"threads_started":1,"wal-append.queue_time_us":244119}
I20260709 13:03:51.739328 18458 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.811601 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.793288 15163 rpcz_store.cc:276] 0709 13:03:49.940689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.940827 (+   138us) service_pool.cc:224] Handling call
0709 13:03:49.941170 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:50.185632 (+244462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.185678 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:50.192275 (+  6597us) log.cc:844] Serialized 52 byte log entry
0709 13:03:50.304775 (+112500us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:50.322930 (+ 18155us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.019745 (+696815us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b1800317fe0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:03:51.019943 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.020016 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.336954 (+316938us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:51.792509 (+455555us) raft_consensus.cc:1685] finished
0709 13:03:51.792540 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.792705 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13775,"prepare.queue_time_us":29314,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":758169,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":11503,"spinlock_wait_cycles":41318528,"thread_start_us":288287,"threads_started":2,"wal-append.queue_time_us":125733}
I20260709 13:03:51.744666 18266 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.815796 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:51.777948 15079 rpcz_store.cc:269] 0709 13:03:49.792712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.792826 (+   114us) service_pool.cc:224] Handling call
0709 13:03:51.643404 (+1850578us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b540014da54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:51.643599 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296040704}
I20260709 13:03:51.819916 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:03:51.825484 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.091s	sys 0.209s
I20260709 13:03:51.827277 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 854) took 2420 ms. Trace:
W20260709 13:03:51.865804 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 770596us
I20260709 13:03:51.778029 18589 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.823899 18590 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.831610 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:51.833485 18540 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.747154 18245 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.842962 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.796217 18582 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:51.836006 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.816133 18311 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.850939 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:51.851878 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.852589 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:51.817625 18543 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.788960 18374 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.886271 18440 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:03:51.852398 18604 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:51.911679 15156 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:51.912997 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.913841 15079 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:51.916169 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:51.830706 18598 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:51.842276 17831 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:03:51.823735 18227 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:51.824100 18308 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.863029 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.832885 15161 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:51.802804 18473 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:51.845441 18517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.424s	user 0.009s	sys 0.008s
I20260709 13:03:51.838872 18580 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.869225 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:51.928946 15232 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:51.928785 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:51.865860 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51003.9us
I20260709 13:03:51.866034 15086 rpcz_store.cc:276] 0709 13:03:49.406791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.422918 (+ 16127us) service_pool.cc:224] Handling call
0709 13:03:49.441414 (+ 18496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.366386 (+924972us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54001a57a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f96206f 00007f0d3f9daf60 00007f0d3f9dc7de 00007f0d3f9e6b49 00007f0d3f9e7416 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62
0709 13:03:51.826942 (+1460556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.827204 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273308032}
I20260709 13:03:51.871835 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:51.872749 15230 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:03:51.930255 15230 pending_rounds.cc:85] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
I20260709 13:03:51.930481 15230 pending_rounds.cc:107] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:03:51.849871 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:51.930581 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58832.5us
I20260709 13:03:51.930711 15230 raft_consensus.cc:2889] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.931124 15085 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 4.
I20260709 13:03:51.834339 18378 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:51.931885 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:51.875371 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:51.843531 18433 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.886739 15228 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:51.887342 15843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 942) took 1537 ms (client timeout 1785 ms). Trace:
W20260709 13:03:51.943666 15843 rpcz_store.cc:269] 0709 13:03:50.349888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.350052 (+   164us) service_pool.cc:224] Handling call
0709 13:03:51.887179 (+1537127us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b540041cdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:51.887288 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935489920}
I20260709 13:03:51.874593 18556 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:51.893142 15844 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:51.890724 18471 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.900410 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:51.902176 18502 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:51.903491 18439 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.903613 18455 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:51.906047 18311 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:51.829282 18605 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:03:51.905615 18582 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.834635 18361 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.908901 18565 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.869115 18538 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:51.909251 18543 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:52.000613 18559 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.001s
I20260709 13:03:51.930738 15086 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:51.833200 15158 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:51.874123 18156 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:51.922513 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.941360 15235 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:51.886328 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:52.020071 15845 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.947376 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:51.961500 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:51.927583 18613 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:03:51.963651 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:51.857321 18381 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.910701 18440 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:51.982959 15162 raft_consensus.cc:2368] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:03:51.936991 18497 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.939296 18227 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:51.992476 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:52.002754 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 102497us
I20260709 13:03:51.940490 18523 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.958079 18369 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:51.893771 18245 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.886539 18540 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.958366 18546 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:51.874737 18590 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:51.980322 18600 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:51.867689 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:52.012104 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 184606us
I20260709 13:03:52.015471 15085 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:03:51.958798 18548 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.020681 15160 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:51.937804 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 178413us
I20260709 13:03:52.021926 15079 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:03:52.021829 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58347.8us
W20260709 13:03:52.022475 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 202639us
I20260709 13:03:51.990463 18597 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:51.954052 18308 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.005532 18615 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:52.013527 18465 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.001s
I20260709 13:03:52.011240 18374 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.056885 18369 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:51.928917 18604 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.031387 18503 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:51.999683 18439 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:51.964306 18433 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.037736 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:52.038944 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 972) took 1646 ms. Trace:
I20260709 13:03:52.116044 15158 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:52.115926 15844 rpcz_store.cc:276] 0709 13:03:50.392120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.392999 (+   879us) service_pool.cc:224] Handling call
0709 13:03:51.887886 (+1494887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.954580 (+ 66694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.038751 (+ 84171us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540041cdd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.038872 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3436971392}
I20260709 13:03:52.014509 18330 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:52.028621 18469 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.586s	user 0.015s	sys 0.000s
I20260709 13:03:52.048134 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.122218 15229 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:52.048264 15080 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:52.024555 18582 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:52.023452 18502 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:51.995491 18471 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:52.053953 15163 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:52.061730 15087 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:52.014995 18612 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:51.966650 18556 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.071841 15157 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:52.068949 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:52.081261 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.083024 15079 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:52.136292 15079 pending_rounds.cc:85] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:03:52.136592 15079 pending_rounds.cc:107] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:52.114645 18458 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:52.086735 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:52.139605 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52959.4us
I20260709 13:03:52.024775 18559 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.049140 18523 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.017690 18565 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.038450 18497 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.074189 18615 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:52.095965 18609 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:52.085546 18618 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:52.053328 18600 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:52.097889 18608 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:03:52.098711 18598 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.099866 18543 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.058465 18528 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.057649 17880 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:52.100586 18589 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.114312 15160 raft_consensus.cc:2410] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:03:52.114667 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 55409.1us
I20260709 13:03:52.130817 15154 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:52.133715 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:52.135802 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 67284.8us
I20260709 13:03:52.114532 18560 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.069489 18548 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.136989 15079 raft_consensus.cc:2889] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.959234 18378 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.135680 15228 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:52.149158 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:52.138386 18509 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.138258 18503 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:03:52.127663 18502 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:52.161504 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 288828us
W20260709 13:03:52.162034 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 112895us
W20260709 13:03:52.162505 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 106394us
I20260709 13:03:52.170608 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:52.134366 18266 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.173964 18627 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:52.133725 18227 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.128067 18469 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.143756 18516 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:52.314247 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.181632 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.276065 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 405264us
W20260709 13:03:52.454986 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.149996 18619 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:52.138789 18374 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:52.148386 18311 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:52.517331 15845 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:52.597944 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.598583 15160 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:52.599426 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:52.599872 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.600131 15160 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:52.260361 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.562274 18571 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.590s	user 0.000s	sys 0.004s
I20260709 13:03:52.606427 18548 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.607349 18503 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:52.607571 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 862) took 2379 ms. Trace:
I20260709 13:03:52.607851 18503 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:52.607966 15082 rpcz_store.cc:276] 0709 13:03:50.227715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.227848 (+   133us) service_pool.cc:224] Handling call
0709 13:03:50.234494 (+  6646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.234859 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.234896 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.257776 (+ 22880us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.361702 (+103926us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:50.361964 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.362030 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.171670 (+1809640us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:52.607299 (+435629us) raft_consensus.cc:1685] finished
0709 13:03:52.607319 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.607515 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61996,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16055,"spinlock_wait_cycles":14017792,"thread_start_us":51334,"threads_started":1,"wal-append.queue_time_us":602835}
W20260709 13:03:52.565860 18587 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.367s	user 0.009s	sys 0.002s
W20260709 13:03:52.548974 18044 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.000s
I20260709 13:03:52.259842 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 902) took 1973 ms. Trace:
I20260709 13:03:52.612445 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2435) took 1444 ms. Trace:
I20260709 13:03:52.603711 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.614665 15079 rpcz_store.cc:276] 0709 13:03:51.167759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.061447 (+893688us) service_pool.cc:224] Handling call
0709 13:03:52.082591 (+ 21144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.611984 (+529393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.612381 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.615943 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.258957 15159 raft_consensus.cc:2368] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:52.601521 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.617286 18616 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:52.610143 18619 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.617733 18321 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:52.618101 18559 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.619298 18619 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.246766 18565 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:52.542936 18523 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.616026 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:52.302325 18555 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.128s	user 0.013s	sys 0.000s
I20260709 13:03:52.611445 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:52.177323 18520 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.079s	user 0.003s	sys 0.010s
W20260709 13:03:52.575059 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 404502us
I20260709 13:03:52.170226 18615 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:52.527115 18540 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.254959 18622 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:52.240329 18624 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:03:52.525694 18618 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:52.610838 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.632061 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:52.635757 15158 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:52.614562 15846 rpcz_store.cc:276] 0709 13:03:50.286736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.287066 (+   330us) service_pool.cc:224] Handling call
0709 13:03:50.287483 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.313713 (+ 26230us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:50.313742 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.336620 (+ 22878us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.406091 (+ 69471us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:50.406244 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.406317 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.404639 (+998322us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:52.257425 (+852786us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b24000d0cb8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c77d9c4 00007f0d3c77bacb 00007f0d3c936dc9 00007f0d3c936a55 00007f0d3c935fb0 00007f0d3f5ac488 00007f0d3f5acbe8 00007f0d3f584398 00007f0d3f57f76a 00007f0d3f57ebb6
0709 13:03:52.259551 (+  2126us) raft_consensus.cc:1685] finished
0709 13:03:52.259582 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.259770 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25781,"raft.queue_time_us":28136,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":16135,"spinlock_wait_cycles":24035328,"thread_start_us":45472,"threads_started":1,"wal-append.queue_time_us":568082}
I20260709 13:03:52.170373 18473 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.622460 15084 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:52.623320 15079 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:52.623441 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:52.623754 15163 raft_consensus.cc:2376] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8. Re-sending same reply.
I20260709 13:03:52.624393 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:52.265658 18465 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.624542 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.245260 18245 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.254626 18604 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.275444 18506 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.625865 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.608682 18612 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:52.520985 18381 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.171887 18580 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:52.649766 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.695827 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.696566 15160 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:03:52.258757 18556 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:52.661275 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.661136 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.694981 15079 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:52.259652 18597 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:52.641052 15082 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:03:52.625684 18512 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.804s	user 0.007s	sys 0.004s
I20260709 13:03:52.176417 18264 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.449s
I20260709 13:03:52.596935 18627 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.644299 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.645310 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.622846 18548 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:52.174829 18590 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.584779 17880 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.610170 18502 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:52.718703 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:52.719655 15203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.945s (SENT) [suppressed 12 similar messages]
I20260709 13:03:52.745114 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:52.636502 18520 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.503276 18361 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:52.720820 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:52.698360 15233 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:52.649547 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:52.722323 15228 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:52.621691 18266 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:03:52.723829 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:52.662931 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:52.724634 15235 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.721503 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2439) took 1400 ms. Trace:
I20260709 13:03:52.793923 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.643468 18636 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:52.625394 18555 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:52.648676 18513 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.637861 18622 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:52.736270 15163 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:52.725462 18471 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.761686 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.608877 18503 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.697342 18629 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:52.697585 18523 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:52.718099 15236 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:52.719985 18630 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:52.729125 18624 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:03:52.794636 15158 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:52.862025 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2565) took 1110 ms. Trace:
I20260709 13:03:52.794340 15079 rpcz_store.cc:276] 0709 13:03:51.321387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.694260 (+1372873us) service_pool.cc:224] Handling call
0709 13:03:52.694628 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.721223 (+ 26595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.721445 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.247782 18369 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.649040 18340 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
W20260709 13:03:52.797559 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 134730us
W20260709 13:03:52.798182 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53236.4us
W20260709 13:03:52.798611 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77896.1us
I20260709 13:03:52.649972 18465 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:52.800853 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:52.800971 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2568) took 1032 ms. Trace:
I20260709 13:03:52.875062 15163 rpcz_store.cc:276] 0709 13:03:51.768915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.643902 (+874987us) service_pool.cc:224] Handling call
0709 13:03:52.800895 (+156993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.876191 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.793190 18556 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.799580 18473 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:52.797698 18560 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.797863 18506 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.803956 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:52.805420 18640 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:52.804747 18513 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.921967 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2544) took 1726 ms. Trace:
I20260709 13:03:52.806223 18517 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:52.922461 15161 rpcz_store.cc:276] 0709 13:03:51.195637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.196318 (+   681us) service_pool.cc:224] Handling call
0709 13:03:52.921740 (+1725422us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.921909 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460166528}
W20260709 13:03:52.808145 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 873) took 1444 ms (client timeout 1677 ms). Trace:
I20260709 13:03:52.808310 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.808482 18627 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.645377 18540 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 13:03:52.810325 17880 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.810475 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 871) took 1447 ms. Trace:
I20260709 13:03:52.650439 18618 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.811400 18598 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:52.812813 18378 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:52.973088 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.814177 18469 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.800225 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:52.963960 18503 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:52.988031 18465 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:52.990466 18520 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.957489 18473 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:52.817247 18642 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:52.991488 18473 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.992089 18642 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:52.992550 18642 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:52.819763 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:52.821653 18582 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
W20260709 13:03:52.822536 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 194300us
W20260709 13:03:52.823510 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2552) took 1622 ms (client timeout 1945 ms). Trace:
W20260709 13:03:52.824162 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 686) took 1502 ms (client timeout 1677 ms). Trace:
I20260709 13:03:52.827332 18497 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.828393 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2441) took 1506 ms. Trace:
I20260709 13:03:52.830848 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.831873 18636 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:52.835235 18574 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:52.835659 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 186164us
I20260709 13:03:53.006129 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.937716 18517 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:52.835304 18502 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:52.765363 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 164449us
I20260709 13:03:52.836160 18549 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:52.778044 18590 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:52.843284 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 682) took 1522 ms. Trace:
I20260709 13:03:52.892603 18434 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:03:53.013487 15154 rpcz_store.cc:276] 0709 13:03:51.320978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.842654 (+521676us) service_pool.cc:224] Handling call
0709 13:03:52.843084 (+1000430us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540037f854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.843202 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587264256}
I20260709 13:03:52.843636 18555 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.847221 18439 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:52.847328 18227 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:52.848089 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2443) took 1525 ms. Trace:
I20260709 13:03:52.737850 18584 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.850667 15088 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:52.852825 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133193us
I20260709 13:03:52.852715 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:52.852265 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:52.719718 18633 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:52.864931 15229 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:52.867699 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2547) took 1667 ms (client timeout 2106 ms). Trace:
W20260709 13:03:53.028832 15162 rpcz_store.cc:269] 0709 13:03:51.199852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.199952 (+   100us) service_pool.cc:224] Handling call
0709 13:03:52.861213 (+1661261us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54002baf54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.867619 (+  6406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525321984}
I20260709 13:03:52.862571 15158 rpcz_store.cc:276] 0709 13:03:51.752109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.778126 (+1026017us) service_pool.cc:224] Handling call
0709 13:03:52.861941 (+ 83815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.871771 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:53.030239 15079 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:53.030925 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:52.876621 15163 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:52.799167 18580 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.918956 15228 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:52.806315 18622 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.860411 18523 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:52.963063 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2431) took 1856 ms. Trace:
W20260709 13:03:52.963522 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 91539.9us
I20260709 13:03:52.026376 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:52.964685 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 679) took 1686 ms (client timeout 1914 ms). Trace:
W20260709 13:03:53.048409 15159 rpcz_store.cc:269] 0709 13:03:51.278027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.829616 (+551589us) service_pool.cc:224] Handling call
0709 13:03:52.964497 (+1134881us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b540013e2d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.964630 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774204800}
I20260709 13:03:53.049876 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2558) took 1839 ms. Trace:
I20260709 13:03:53.050262 15160 rpcz_store.cc:276] 0709 13:03:51.210223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.852426 (+1642203us) service_pool.cc:224] Handling call
0709 13:03:53.049815 (+197389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.014490 18549 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:52.965768 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1814) took 1889 ms (client timeout 1585 ms). Trace:
I20260709 13:03:52.966423 15086 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:03:52.967576 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 670) took 3213 ms (client timeout 2415 ms). Trace:
W20260709 13:03:53.061573 15155 rpcz_store.cc:269] 0709 13:03:49.743131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.743248 (+   117us) service_pool.cc:224] Handling call
0709 13:03:52.956263 (+3213015us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002878d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.956468 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990632320}
I20260709 13:03:52.866680 18629 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:52.967864 15234 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:52.965803 15082 rpcz_store.cc:276] 0709 13:03:51.363117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.809665 (+1446548us) service_pool.cc:224] Handling call
0709 13:03:52.809974 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.810143 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.810171 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.810214 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.810243 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.810296 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.810422 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.985955 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.063305 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.063925 15079 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:52.995203 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.934173 18471 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:53.001398 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 170637us
I20260709 13:03:53.000509 15084 rpcz_store.cc:276] 0709 13:03:51.321941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.841768 (+519827us) service_pool.cc:224] Handling call
0709 13:03:52.828155 (+986387us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54003efdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.828320 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586241792}
W20260709 13:03:53.002007 15157 rpcz_store.cc:269] 0709 13:03:51.200968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.831834 (+630866us) service_pool.cc:224] Handling call
0709 13:03:52.823327 (+991493us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400187ed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.823451 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425845504}
W20260709 13:03:53.002691 15156 rpcz_store.cc:269] 0709 13:03:51.322175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.768553 (+446378us) service_pool.cc:224] Handling call
0709 13:03:52.824013 (+1055460us) spinlock_profiling.cc:243] Waited 70.1 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.824108 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161249152}
W20260709 13:03:52.968556 15087 rpcz_store.cc:269] 0709 13:03:51.363744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.368619 (+  4875us) service_pool.cc:224] Handling call
0709 13:03:52.807963 (+1439344us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b540011a3d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.808080 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706775808}
I20260709 13:03:53.008960 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:53.010854 15842 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:53.070011 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.011368 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.071455 15846 raft_consensus.cc:2410] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:53.072633 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.073431 15846 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:53.012104 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.012416 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.018378 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.018759 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.077878 15839 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:52.927322 18556 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.968536 18506 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.081167 18506 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.023909 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.020048 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:53.068588 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59723.8us
I20260709 13:03:53.025990 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.082861 15154 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:53.084079 15154 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:53.084596 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2549) took 1884 ms. Trace:
I20260709 13:03:53.084926 15154 rpcz_store.cc:276] 0709 13:03:51.200341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.083482 (+1883141us) service_pool.cc:224] Handling call
0709 13:03:53.083774 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.084385 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.084546 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.026055 15080 rpcz_store.cc:276] 0709 13:03:51.322442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.811337 (+488895us) service_pool.cc:224] Handling call
0709 13:03:52.847925 (+1036588us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540039b554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.848025 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440752384}
I20260709 13:03:53.033353 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:53.038143 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 185956us
I20260709 13:03:53.097087 15848 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:53.097160 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.098474 15844 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:53.038807 15237 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:53.101030 18506 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:53.101715 18506 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:53.101954 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:53.037750 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.037674 18520 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:53.103143 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:53.103466 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.956441 18560 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:52.965328 15232 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:53.104032 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:53.047417 15085 rpcz_store.cc:276] 0709 13:03:51.106704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.931732 (+825028us) service_pool.cc:224] Handling call
0709 13:03:51.932029 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.057327 (+125298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.870363 (+813036us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:52.963003 (+ 92640us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191336320}
I20260709 13:03:53.105389 18330 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:53.105686 15085 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:53.105917 18330 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:53.106009 18560 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.016891 18616 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:53.107124 18616 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:53.106922 18330 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.044987 18598 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:53.054478 15233 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:53.056279 18439 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.056509 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.109732 15080 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:53.109586 18439 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.049643 18580 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.112823 15845 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 4.
I20260709 13:03:53.113790 18580 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.058419 18644 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:53.114491 18644 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.115154 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.116372 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:52.870913 18630 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.063390 15161 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:53.118134 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.118563 15161 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 7. Ops: []
I20260709 13:03:53.118846 18630 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.119411 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2554) took 1917 ms. Trace:
I20260709 13:03:53.119848 15161 rpcz_store.cc:276] 0709 13:03:51.201530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.118010 (+1916480us) service_pool.cc:224] Handling call
0709 13:03:53.118353 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.119119 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.119344 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.121392 15161 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:03:53.121773 18556 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.120543 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:52.733454 18597 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:53.060627 15083 rpcz_store.cc:269] 0709 13:03:51.076491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.091601 (+ 15110us) service_pool.cc:224] Handling call
0709 13:03:52.965561 (+1873960us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54003cf354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:52.965700 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067130752}
I20260709 13:03:53.123198 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.065663 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.124396 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.067039 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:53.124935 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:53.125530 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58577.1us
I20260709 13:03:53.126468 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:53.127796 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:53.128059 18455 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:53.019363 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.107514 18616 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:53.129189 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:53.129889 18614 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:53.130796 18614 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:53.128624 18455 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.131284 18614 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.131892 18614 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.036965 18584 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.109928 15159 raft_consensus.cc:2368] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:53.133020 18633 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:53.133175 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.133719 18633 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.134073 18633 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.134393 18633 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.135035 18556 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:53.137491 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.138134 15087 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:53.138980 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.108390 18598 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.124977 15083 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 4.
I20260709 13:03:53.139737 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:53.057111 18473 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.103690 15840 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.142836 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.137832 18616 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.144042 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.145159 18616 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.147626 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.148594 15842 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:53.149569 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.150298 15842 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:53.151307 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.151902 15842 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.122071 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2557) took 1912 ms. Trace:
I20260709 13:03:53.153508 15161 rpcz_store.cc:276] 0709 13:03:51.209884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.120735 (+1910851us) service_pool.cc:224] Handling call
0709 13:03:53.121113 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.121835 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.122025 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.158339 15161 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:53.142730 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.159448 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 678) took 1944 ms. Trace:
I20260709 13:03:53.144606 15232 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:03:53.159999 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.160851 15228 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:53.161876 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.118690 15839 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:53.163635 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.164343 15839 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:53.165652 15839 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:53.154886 18473 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.155596 18465 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.166779 18473 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.166917 18465 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.138309 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:53.169965 18630 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:53.172151 18630 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:53.159823 15161 rpcz_store.cc:276] 0709 13:03:51.215233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.157760 (+1942527us) service_pool.cc:224] Handling call
0709 13:03:53.158112 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.159021 (+   909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.159382 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.139750 15087 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:53.160852 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.179644 15232 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:53.180776 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:53.186990 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 163147us
I20260709 13:03:53.189242 15840 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:53.192386 15156 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:53.193447 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2548) took 1993 ms. Trace:
I20260709 13:03:53.193918 15156 rpcz_store.cc:276] 0709 13:03:51.200133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.191607 (+1991474us) service_pool.cc:224] Handling call
0709 13:03:53.192067 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.193030 (+   963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.193345 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.190685 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:53.066658 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.200759 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:53.182065 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:53.186192 15232 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:53.201719 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 683) took 1880 ms. Trace:
I20260709 13:03:53.027637 18641 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:03:53.207199 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 89907us
I20260709 13:03:53.179955 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:53.233662 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.243790 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.244024 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.244679 15082 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:53.208743 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.051546 18574 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:53.252556 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.254952 15234 raft_consensus.cc:2410] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:53.195225 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:53.257938 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57271.3us
I20260709 13:03:53.259173 15814 leader_election.cc:400] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:53.259601 15814 leader_election.cc:403] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:53.207764 15128 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.585s (SENT)
I20260709 13:03:53.262398 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:53.264020 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56295.7us
I20260709 13:03:53.266335 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.267374 15234 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:53.268126 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:53.207347 15155 rpcz_store.cc:276] 0709 13:03:51.321387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.066423 (+1745036us) service_pool.cc:224] Handling call
0709 13:03:53.201643 (+135220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.052057 18523 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.252451 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.270470 15084 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:53.252893 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.271200 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 872) took 1907 ms. Trace:
W20260709 13:03:53.253561 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.020253 18590 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.271582 15084 rpcz_store.cc:276] 0709 13:03:51.363452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.206749 (+1843297us) service_pool.cc:224] Handling call
0709 13:03:53.266455 (+ 59706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.270895 (+  4440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.271136 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.062867 18549 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.025367 18555 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.265702 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:53.257843 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:53.206920 18321 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:53.241032 18580 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:53.270155 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.282860 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:53.284466 15079 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:53.282178 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.284524 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.290951 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:53.294544 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:53.295861 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:53.298534 15842 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:53.221660 18630 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:53.299141 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.263029 18612 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:53.260700 18433 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.307165 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.316295 15232 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:53.310609 15157 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:53.273643 18556 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:53.277642 18538 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:53.318545 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.319077 18512 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:53.261742 18613 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:03:53.207198 18471 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.333472 18471 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.334635 15161 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:53.181493 18635 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:03:53.294584 18264 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8.
I20260709 13:03:53.321825 18556 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.338245 18556 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.340570 18631 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:53.303494 18361 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:53.343472 18361 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:53.343883 18361 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.344131 18502 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.270395 17880 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:03:53.345851 17880 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:53.346211 17880 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:53.347045 17880 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.348403 17880 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:53.303164 18548 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.349435 18548 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.349982 18548 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.350919 18590 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.352828 18361 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.356365 18361 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.349392 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.359059 18439 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.359423 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.360204 15842 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.268908 18506 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:53.265455 18633 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.325680 18512 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.253363 18641 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.407440 18641 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.408382 18512 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:53.413120 15842 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:53.294888 18528 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:03:53.414557 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.419092 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:53.372498 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.427321 15162 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:53.388523 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.428548 15157 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:03:53.310104 18516 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.005s
I20260709 13:03:53.013696 18636 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:53.392823 18645 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:53.437186 15081 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:03:53.437649 15081 pending_rounds.cc:85] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:03:53.437906 15081 pending_rounds.cc:107] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:03:53.315701 18321 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:53.373158 18556 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.368219 18548 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.402658 18555 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.415108 18528 consensus_queue.cc:1243] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:53.396329 18615 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.751s	user 0.001s	sys 0.006s
I20260709 13:03:53.447184 18636 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:53.438184 15081 raft_consensus.cc:2889] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:53.373755 18618 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.436285 18582 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.456084 18513 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.420679 18473 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.404906 18633 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:53.464505 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 468939us
I20260709 13:03:53.400382 18614 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.475395 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.521631 15157 raft_consensus.cc:2410] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:03:53.481321 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:53.522850 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 680) took 2211 ms. Trace:
I20260709 13:03:53.523557 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.523459 15158 rpcz_store.cc:276] 0709 13:03:51.311501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.500648 (+2189147us) service_pool.cc:224] Handling call
0709 13:03:53.522072 (+ 21424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.522357 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.522400 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.522470 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:53.522531 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.522618 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.522785 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.528101 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.483991 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2432) took 2377 ms. Trace:
I20260709 13:03:53.529398 15086 rpcz_store.cc:276] 0709 13:03:51.106991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.024983 (+917992us) service_pool.cc:224] Handling call
0709 13:03:52.966173 (+941190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.061578 (+ 95405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.483756 (+422178us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54003cf350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:53.483921 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132467072}
W20260709 13:03:53.558964 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77719.6us
I20260709 13:03:53.560977 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:53.561447 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.500339 18588 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.570994 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:53.371814 18629 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:53.574146 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 316368us
I20260709 13:03:53.580972 15228 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:53.579599 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.556450 18633 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:53.489416 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.587538 15162 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:53.587972 15231 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:03:53.494838 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.588539 15231 pending_rounds.cc:85] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 2
I20260709 13:03:53.501426 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:53.578567 18647 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:03:53.503549 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 186199us
W20260709 13:03:53.528863 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.286882 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:53.490916 18615 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.565586 18528 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.562609 18560 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:53.436534 18503 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:53.501194 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.589097 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.605960 15845 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:53.605997 15158 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:53.584201 18618 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:53.589258 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:53.609351 15157 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:53.590652 15160 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:53.502185 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.590937 15231 pending_rounds.cc:107] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:03:53.510474 18582 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.594099 18473 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:53.507040 18439 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:03:53.631080 15160 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:53.600911 18330 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:53.563601 18590 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:53.607589 18506 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:53.599807 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:53.589118 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 94401.8us
I20260709 13:03:53.558331 18574 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:53.646267 18330 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.621220 15231 raft_consensus.cc:2889] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:53.596117 18264 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:53.624049 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.575911 18588 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.629639 15233 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:53.585808 18321 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.605122 18528 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:53.598444 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 311647us
I20260709 13:03:53.697458 15233 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:53.699419 15233 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:53.702842 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.703500 15845 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:53.644649 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 684) took 2323 ms. Trace:
I20260709 13:03:53.704700 15160 rpcz_store.cc:276] 0709 13:03:51.321687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.630307 (+2308620us) service_pool.cc:224] Handling call
0709 13:03:53.630662 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.631609 (+   947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.644576 (+ 12967us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.483376 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.601547 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:53.593240 18633 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.619778 18618 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:53.670481 15846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.013s	sys 0.021s
W20260709 13:03:53.673164 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73445.1us
I20260709 13:03:53.619234 18503 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:53.672474 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 413469us
I20260709 13:03:53.674250 15163 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:53.674629 15157 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:53.717938 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 685) took 2396 ms. Trace:
I20260709 13:03:53.717267 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:53.671615 15843 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.004s	sys 0.011s
I20260709 13:03:53.683240 15079 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:53.652948 18439 consensus_queue.cc:1243] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:53.694818 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:53.640396 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:53.673854 18556 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.709693 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.677021 18330 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:53.714718 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 186704us
W20260709 13:03:53.761121 15843 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.004s	sys 0.013s
I20260709 13:03:53.718559 15163 rpcz_store.cc:276] 0709 13:03:51.321926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.647542 (+2325616us) service_pool.cc:224] Handling call
0709 13:03:53.673843 (+ 26301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.717586 (+ 43743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.717857 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.620484 18600 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:53.693799 18538 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:53.703061 18227 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.002s
I20260709 13:03:53.762267 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:53.786763 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.703464 18528 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.720644 15160 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:03:53.760531 15846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.014s	sys 0.021s
I20260709 13:03:53.768285 15158 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:03:53.768107 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 50949.5us
I20260709 13:03:53.768821 15845 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:53.792235 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:53.769688 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2016) took 1897 ms. Trace:
I20260709 13:03:53.795717 15163 rpcz_store.cc:276] 0709 13:03:51.872392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.768520 (+1896128us) service_pool.cc:224] Handling call
0709 13:03:53.768940 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.769200 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.769246 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.769309 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:53.769365 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.769450 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.769608 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.800253 15163 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:53.708026 18473 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:03:53.780069 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 880) took 1010 ms. Trace:
I20260709 13:03:53.815863 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2560) took 2093 ms. Trace:
I20260709 13:03:53.783814 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.817025 15079 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:53.817414 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.711918 18548 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.794915 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:53.829778 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.830807 15846 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:53.708129 15840 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:53.771512 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:53.832651 15839 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.815007 18471 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:53.819943 18659 raft_consensus.cc:670] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } [suppressed 4 similar messages]
W20260709 13:03:53.820134 18656 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:03:53.821331 18658 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:03:53.866128 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.867244 15079 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:53.823045 18613 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:53.823997 18618 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:53.824306 18642 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.831s	user 0.006s	sys 0.003s
W20260709 13:03:53.815258 18648 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:03:53.884295 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.816397 15160 rpcz_store.cc:276] 0709 13:03:51.722323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.710454 (+1988131us) service_pool.cc:224] Handling call
0709 13:03:53.720328 (+  9874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.815530 (+ 95202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.815802 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.831488 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:53.816591 18369 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:53.885743 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54361.4us
I20260709 13:03:53.887964 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:53.888209 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.815964 15083 rpcz_store.cc:276] 0709 13:03:52.769535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.140864 (+371329us) service_pool.cc:224] Handling call
0709 13:03:53.141178 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.141367 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:53.478055 (+336688us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.779610 (+301555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.779672 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.779730 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.779799 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.780011 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":438775,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":71,"thread_start_us":637541,"threads_started":2}
I20260709 13:03:53.855403 18433 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.858176 18600 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.835879 18471 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:53.874320 18361 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:53.855118 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 160418us
I20260709 13:03:53.901458 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.875798 18590 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:53.822288 18264 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.913275 18264 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.913743 18308 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.914569 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2445) took 1096 ms. Trace:
W20260709 13:03:53.915300 18512 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.001s
I20260709 13:03:53.915622 18264 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.915740 18512 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.901316 18361 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.901631 18503 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.917423 18361 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.713069 18660 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 13:03:53.782868 18440 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.784689 18538 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.817334 18621 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.888957 15079 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:53.827311 18614 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:53.847142 18615 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:53.805788 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 691) took 1728 ms. Trace:
I20260709 13:03:53.937875 15163 rpcz_store.cc:276] 0709 13:03:52.077772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.799342 (+1721570us) service_pool.cc:224] Handling call
0709 13:03:53.799781 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.805343 (+  5562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.805687 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.905000 18469 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:53.848611 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53833.2us
I20260709 13:03:53.896961 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2448) took 1077 ms. Trace:
W20260709 13:03:53.952759 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.951946 15082 rpcz_store.cc:276] 0709 13:03:52.819183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.245878 (+426695us) service_pool.cc:224] Handling call
0709 13:03:53.255837 (+  9959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.256091 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:53.718475 (+462384us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.896461 (+177986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.896539 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.896604 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.896699 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.896884 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16525,"prepare.queue_time_us":329961,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":9030,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":18049,"thread_start_us":588856,"threads_started":2}
W20260709 13:03:53.855747 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 69079.7us
I20260709 13:03:53.954752 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.898710 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2572) took 1025 ms. Trace:
I20260709 13:03:53.712062 18580 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:03:53.906538 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:53.897670 18433 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:53.901999 18582 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:53.902149 18471 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.910856 18588 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:53.912360 17880 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 13:03:53.914978 15087 rpcz_store.cc:276] 0709 13:03:52.818396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.177413 (+359017us) service_pool.cc:224] Handling call
0709 13:03:53.177850 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:53.178828 (+   978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.178856 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:53.181638 (+  2782us) log.cc:844] Serialized 52 byte log entry
0709 13:03:53.475863 (+294225us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:53.855041 (+379178us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.859280 (+  4239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.859413 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.914315 (+ 54902us) raft_consensus.cc:1685] finished
0709 13:03:53.914336 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.914503 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8518,"prepare.queue_time_us":144306,"prepare.run_cpu_time_us":4153,"prepare.run_wall_time_us":15251,"raft.queue_time_us":4684,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":10834,"thread_start_us":571236,"threads_started":2,"wal-append.queue_time_us":425643}
I20260709 13:03:53.921823 15839 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:03:53.957726 18654 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:03:53.895159 18627 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:53.865921 18378 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:03:53.935446 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:53.935573 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2559) took 2725 ms. Trace:
I20260709 13:03:53.939692 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1833) took 1241 ms. Trace:
I20260709 13:03:53.939839 15163 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:03:53.875792 18652 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:03:53.949796 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:53.884505 18613 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.956112 15160 rpcz_store.cc:276] 0709 13:03:52.873598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.885426 (+1011828us) service_pool.cc:224] Handling call
0709 13:03:53.898188 (+ 12762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.898364 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.898388 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.898435 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:53.898468 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.898521 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.898616 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.933967 18439 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:03:53.958148 18676 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:53.884728 18642 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:53.927299 18440 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:53.935319 18538 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.993944 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.031359 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.032280 15839 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:03:53.891031 18369 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.033244 18369 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.996327 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:54.002764 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 114826us
I20260709 13:03:54.042446 15087 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:54.042857 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:53.969458 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63052.3us
I20260709 13:03:54.044350 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:54.015697 18633 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.008358 15159 rpcz_store.cc:276] 0709 13:03:51.210445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.133491 (+1923046us) service_pool.cc:224] Handling call
0709 13:03:53.133844 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.134181 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.134209 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.134659 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.498696 (+364037us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.710214 (+211518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.710438 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.710510 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.935278 (+224768us) raft_consensus.cc:1685] finished
0709 13:03:53.935310 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.935495 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20831,"prepare.queue_time_us":98429,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2228,"raft.queue_time_us":783,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":4585,"thread_start_us":551495,"threads_started":2,"wal-append.queue_time_us":465855}
I20260709 13:03:53.987493 18433 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.971771 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.060530 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:53.986855 18473 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.008968 15236 rpcz_store.cc:276] 0709 13:03:52.698255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.878104 (+179849us) service_pool.cc:224] Handling call
0709 13:03:52.878745 (+   641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.889577 (+ 10832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.889625 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.985551 (+ 95926us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.186110 (+200559us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:53.186148 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.186215 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.939408 (+753193us) raft_consensus.cc:1685] finished
0709 13:03:53.939438 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.939618 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10714,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":194290,"threads_started":1,"wal-append.queue_time_us":529396}
I20260709 13:03:53.989830 18469 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:54.026705 18440 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:53.994117 18666 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:03:54.075647 15159 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:54.077504 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 882) took 1148 ms. Trace:
I20260709 13:03:54.081248 15079 rpcz_store.cc:276] 0709 13:03:52.928933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.934650 (+1005717us) service_pool.cc:224] Handling call
0709 13:03:53.969376 (+ 34726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.969720 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:53.995426 (+ 25706us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.077138 (+ 81712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.077198 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:54.077250 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.077316 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.077444 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30046,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":56,"raft.queue_time_us":2496,"thread_start_us":80978,"threads_started":1}
I20260709 13:03:54.089170 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.090837 18465 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:54.092278 18678 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:03:54.095690 18671 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:03:54.095618 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1826) took 1434 ms. Trace:
I20260709 13:03:54.096153 15230 rpcz_store.cc:276] 0709 13:03:52.661671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.661836 (+   165us) service_pool.cc:224] Handling call
0709 13:03:52.662223 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:52.732374 (+ 70151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.732404 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:52.762711 (+ 30307us) log.cc:844] Serialized 52 byte log entry
0709 13:03:53.591734 (+829023us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b18004814e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:03:53.777199 (+185465us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:53.777250 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.777339 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.095292 (+317953us) raft_consensus.cc:1685] finished
0709 13:03:54.095311 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.095554 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70512,"raft.queue_time_us":63038,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":10060,"spinlock_wait_cycles":22930816,"thread_start_us":185230,"threads_started":1,"wal-append.queue_time_us":1248880}
I20260709 13:03:54.106084 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.110669 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.120783 15839 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
W20260709 13:03:54.079847 18523 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.790s	user 0.006s	sys 0.006s
I20260709 13:03:54.128553 15079 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:54.139911 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.140512 15079 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:54.142772 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 98483.8us
I20260709 13:03:54.143801 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.144752 18439 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:54.146021 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 102275us
I20260709 13:03:54.129429 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.155568 15159 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:54.081032 18627 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:54.156536 15079 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:54.130230 18473 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:54.158660 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.161010 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.136128 18465 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:54.136435 18636 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.137259 18633 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:54.177003 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.169011 18685 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:54.169886 18695 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:54.169615 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.171226 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.148145 18512 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:54.174849 18680 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
I20260709 13:03:54.174108 18469 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:54.155511 18666 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:54.178085 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.147743 18439 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.187783 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 700) took 1178 ms. Trace:
I20260709 13:03:54.187980 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.189934 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.193020 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.161935 18473 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.193959 15087 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.149791 18523 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.225225 18523 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.225858 15841 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.184571 18636 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.194463 15079 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:54.228681 18636 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.229549 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.196285 18361 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.230340 15079 raft_consensus.cc:2368] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:54.231084 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.231768 18361 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.232017 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.232560 15232 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:54.196395 18465 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:54.237510 18465 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.237936 18465 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.239557 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.240635 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.220466 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 681) took 2907 ms. Trace:
I20260709 13:03:54.241876 15157 rpcz_store.cc:276] 0709 13:03:51.313311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.610236 (+2296925us) service_pool.cc:224] Handling call
0709 13:03:53.644659 (+ 34423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.741294 (+ 96635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.220126 (+478832us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.220383 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68486016}
I20260709 13:03:54.194290 15161 rpcz_store.cc:276] 0709 13:03:53.009003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.918831 (+909828us) service_pool.cc:224] Handling call
0709 13:03:53.919097 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.919265 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:53.919495 (+   230us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.187352 (+267857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.187483 (+   131us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.187542 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.187607 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.187730 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37557,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":35240,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6761,"thread_start_us":259229,"threads_started":2,"wal-append.queue_time_us":131677}
I20260709 13:03:54.195952 18439 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.202168 18642 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.202605 18684 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:54.184993 18695 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:54.203347 18627 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.206965 18330 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.218029 18613 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.221869 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.222436 18327 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:54.222978 18549 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.929s	user 0.011s	sys 0.001s
W20260709 13:03:54.262660 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.245600 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1828) took 1551 ms. Trace:
I20260709 13:03:54.246259 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.263269 15237 rpcz_store.cc:276] 0709 13:03:52.694628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.695466 (+   838us) service_pool.cc:224] Handling call
0709 13:03:52.727079 (+ 31613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.100465 (+373386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.245370 (+1144905us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000ac8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.245547 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716555136}
I20260709 13:03:54.246346 18556 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.250702 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.191861 18696 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.199232 18473 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.269559 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.190289 18633 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:54.261643 18631 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.920s	user 0.013s	sys 0.000s
I20260709 13:03:54.266050 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2450) took 1446 ms. Trace:
I20260709 13:03:54.265707 15087 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:54.265643 15079 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:54.268994 15082 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.252611 18684 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.264972 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.183398 18369 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.269297 18635 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:54.254503 18695 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.261224 18327 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:54.272997 15086 rpcz_store.cc:276] 0709 13:03:52.819711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.530596 (+710885us) service_pool.cc:224] Handling call
0709 13:03:53.531045 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.713081 (+182036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.713921 (+   840us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.714444 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.871589 (+157145us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:53.871650 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.871759 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.174175 (+302416us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:54.265737 (+ 91562us) raft_consensus.cc:1685] finished
0709 13:03:54.265752 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.265987 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31924,"raft.queue_time_us":68053,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":6896,"spinlock_wait_cycles":21693824,"thread_start_us":240673,"threads_started":2,"wal-append.queue_time_us":294775}
I20260709 13:03:54.282063 15157 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:54.272344 18696 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.289680 18696 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.290374 18433 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.291364 18327 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.292944 15160 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:54.274639 18631 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:54.292465 18433 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.296351 18597 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.297621 18330 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.298277 18528 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:54.284242 18471 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.301993 18528 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.302764 15163 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:54.292492 18327 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:54.267004 18549 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.271370 18556 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.282485 18684 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:54.448776 18697 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:03:54.452986 18633 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.454794 18465 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.453763 18471 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.451185 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 881) took 1571 ms. Trace:
I20260709 13:03:54.473438 15083 rpcz_store.cc:276] 0709 13:03:52.879513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.896126 (+1016613us) service_pool.cc:224] Handling call
0709 13:03:53.923941 (+ 27815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.926265 (+  2324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.926308 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.926907 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.142731 (+215824us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.143480 (+   749us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.151029 (+  7549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.151119 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.450908 (+299789us) raft_consensus.cc:1685] finished
0709 13:03:54.450935 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.451116 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7179,"prepare.queue_time_us":195,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":12503,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":45163,"thread_start_us":172989,"threads_started":1,"wal-append.queue_time_us":320673}
I20260709 13:03:54.482573 18440 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.484006 15161 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.286470 18635 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.531481 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 879) took 1795 ms. Trace:
I20260709 13:03:54.532215 15161 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:54.531625 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 864) took 4140 ms. Trace:
I20260709 13:03:54.532068 15080 rpcz_store.cc:276] 0709 13:03:52.736143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.110628 (+374485us) service_pool.cc:224] Handling call
0709 13:03:53.110880 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.305813 (+194933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.305857 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.305930 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:53.305977 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.531192 (+1225215us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54000db6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.531255 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.531407 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447757056}
I20260709 13:03:54.533048 15081 rpcz_store.cc:276] 0709 13:03:50.391221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.391583 (+   362us) service_pool.cc:224] Handling call
0709 13:03:50.421256 (+ 29673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.490772 (+3069516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.531409 (+1040637us) spinlock_profiling.cc:243] Waited 3.02 s on lock 0x7b540025ba54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.531573 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6934850304}
W20260709 13:03:54.537081 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.541442 18702 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:54.542891 18635 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.543689 18556 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.544409 18473 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:54.546702 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.541508 18700 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:03:54.550710 18538 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.000s
I20260709 13:03:54.545593 18440 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.549455 18465 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.545266 18439 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.570044 18584 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:54.571267 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.581426 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 715) took 1301 ms. Trace:
I20260709 13:03:54.581897 15231 rpcz_store.cc:276] 0709 13:03:53.279956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.693211 (+413255us) service_pool.cc:224] Handling call
0709 13:03:53.693556 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.719600 (+ 26044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.719628 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.719675 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.719712 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.581120 (+861408us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b54002d4954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.581167 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.581309 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59439488}
W20260709 13:03:54.577543 18621 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.002s
I20260709 13:03:54.586784 18549 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:54.591724 18600 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.689s	user 0.004s	sys 0.004s
I20260709 13:03:54.584388 18369 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.593371 18455 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:54.599032 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.599697 18439 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.600709 18264 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:54.601181 18264 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:54.602116 18264 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.603205 18264 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.564515 18538 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:54.605856 18687 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.509s	user 0.002s	sys 0.002s
W20260709 13:03:54.606530 15088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2427) took 3501 ms (client timeout 3314 ms). Trace:
I20260709 13:03:54.606596 18538 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.592700 18621 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.607343 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 694) took 1982 ms. Trace:
I20260709 13:03:54.607717 15158 rpcz_store.cc:276] 0709 13:03:52.625861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.846032 (+1220171us) service_pool.cc:224] Handling call
0709 13:03:53.880394 (+ 34362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.880872 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.880914 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.892021 (+ 11107us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.022560 (+130539us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:54.022607 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.022679 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.605307 (+582628us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:54.607080 (+  1773us) raft_consensus.cc:1685] finished
0709 13:03:54.607099 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.607290 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7856,"spinlock_wait_cycles":21397888,"thread_start_us":88245,"threads_started":1,"wal-append.queue_time_us":151831}
W20260709 13:03:54.607021 15088 rpcz_store.cc:269] 0709 13:03:51.105584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.209079 (+103495us) service_pool.cc:224] Handling call
0709 13:03:54.606321 (+3397242us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000992d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.606438 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485691136}
I20260709 13:03:54.599767 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 971) took 1193 ms. Trace:
I20260709 13:03:54.610723 15843 rpcz_store.cc:276] 0709 13:03:53.406505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.834128 (+427623us) service_pool.cc:224] Handling call
0709 13:03:53.834437 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.041799 (+207362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.041851 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.041906 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.041955 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.599537 (+557582us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54006dc254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.599584 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.599707 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475555200}
I20260709 13:03:54.573228 18517 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.613372 18517 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:54.614782 18701 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:54.619562 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2018) took 1462 ms. Trace:
I20260709 13:03:54.619738 18473 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.619994 15163 rpcz_store.cc:276] 0709 13:03:53.156830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.010728 (+853898us) service_pool.cc:224] Handling call
0709 13:03:54.011135 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.303223 (+292088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.619318 (+316095us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400287dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.619504 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670114688}
I20260709 13:03:54.621505 18559 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:54.621933 18559 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:54.622718 18559 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.623952 18559 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:54.630162 18684 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:03:54.632634 15159 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:03:54.630868 18668 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.789s	user 0.000s	sys 0.013s
I20260709 13:03:54.634428 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2007) took 1494 ms. Trace:
I20260709 13:03:54.634990 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 878) took 2029 ms. Trace:
I20260709 13:03:54.634838 15228 rpcz_store.cc:276] 0709 13:03:53.140072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.161625 (+ 21553us) service_pool.cc:224] Handling call
0709 13:03:54.634225 (+1472600us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54002148d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.634351 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949566208}
I20260709 13:03:54.638660 18378 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:54.639110 15842 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:03:54.639222 18378 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.640482 18684 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.640581 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 959) took 1618 ms. Trace:
I20260709 13:03:54.644044 15841 rpcz_store.cc:276] 0709 13:03:53.022395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.022536 (+   141us) service_pool.cc:224] Handling call
0709 13:03:53.022842 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.226233 (+1203391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.640361 (+414128us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540083e5d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.640529 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766306560}
I20260709 13:03:54.641295 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1830) took 1944 ms. Trace:
I20260709 13:03:54.645049 15229 rpcz_store.cc:276] 0709 13:03:52.697286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.697580 (+   294us) service_pool.cc:224] Handling call
0709 13:03:54.641133 (+1943553us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b5400276fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.641244 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213645568}
I20260709 13:03:54.641438 18378 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.638193 15085 rpcz_store.cc:276] 0709 13:03:52.605989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.106519 (+500530us) service_pool.cc:224] Handling call
0709 13:03:53.106817 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.311468 (+204651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.311512 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.312063 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.671195 (+359132us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.671256 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.219461 (+548205us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400305f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.219538 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.596215 (+376677us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:54.634793 (+ 38578us) raft_consensus.cc:1685] finished
0709 13:03:54.634813 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.634937 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":832,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":8221,"spinlock_wait_cycles":476499072,"thread_start_us":231999,"threads_started":1,"wal-append.queue_time_us":329451}
I20260709 13:03:54.643074 15844 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:54.647773 18667 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.792s	user 0.008s	sys 0.004s
I20260709 13:03:54.651049 18378 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.651576 18378 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.629588 18633 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.653918 18378 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:54.658022 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.09808e+06us
I20260709 13:03:54.659206 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.659967 15841 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:54.653687 18633 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.661289 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.662336 18327 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:54.662588 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:54.660941 18684 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.663563 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.664199 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.685730 15237 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:54.665465 15232 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:54.666442 18635 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:54.667326 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.669399 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.687505 15230 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.688086 15845 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:54.669462 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.670537 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.677488 15229 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:54.683899 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:54.682238 18631 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.685351 18548 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:03:54.693889 18548 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:54.694263 18548 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:54.694366 18528 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:54.694761 18528 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:54.695169 18548 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.695493 18528 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.691022 18644 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:54.697501 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:54.682416 18633 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.637003 18695 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.701406 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.707386 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.708223 15230 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:54.711723 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.698403 18635 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.698635 18548 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.717212 15234 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:54.702190 18330 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.707681 18528 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.719702 18361 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:54.720211 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.725581 18361 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:54.725787 15156 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.726008 18361 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:54.727134 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.727526 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.717720 18633 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:54.729133 18633 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:54.729192 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.712630 18621 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:54.730020 18621 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.730170 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.730408 18621 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.732141 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2020) took 1570 ms. Trace:
I20260709 13:03:54.732518 15160 rpcz_store.cc:276] 0709 13:03:53.161723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.025077 (+863354us) service_pool.cc:224] Handling call
0709 13:03:54.048311 (+ 23234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.294604 (+246293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.731913 (+437309us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400378054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.732083 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562100608}
I20260709 13:03:54.736280 18695 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.729609 18633 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.721320 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.697013 18684 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.746954 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.740203 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 994) took 1540 ms. Trace:
I20260709 13:03:54.757444 15842 rpcz_store.cc:276] 0709 13:03:53.199548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.414181 (+214633us) service_pool.cc:224] Handling call
0709 13:03:54.740060 (+1325879us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54002a12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.740153 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2790486784}
I20260709 13:03:54.749683 15161 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:54.758328 15847 raft_consensus.cc:1217] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-7.7]   Dedup: 2.5->[5.6-7.7]
W20260709 13:03:54.754094 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.758407 18684 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:54.751188 18655 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.037s	user 0.000s	sys 0.013s
I20260709 13:03:54.758930 15847 pending_rounds.cc:85] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 5
I20260709 13:03:54.762436 15847 pending_rounds.cc:107] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:03:54.762255 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.762694 15847 raft_consensus.cc:2889] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.763979 15840 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.760584 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.766858 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.743014 15156 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:03:54.768712 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.767840 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.770025 15163 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:54.771831 15128 leader_election.cc:400] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:54.775800 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.777419 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.777995 15160 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.779541 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:54.780457 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.781258 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.780680 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.782780 15163 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:54.781234 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.783401 15159 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:54.781931 15161 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:54.756515 18361 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.784992 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.785184 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.786312 15230 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:54.786973 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.787439 15230 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:54.787572 15236 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:03:54.788928 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.789521 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.790165 15086 raft_consensus.cc:2368] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:54.790263 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.789031 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.790828 15088 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.792253 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.792799 15079 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.790477 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.793906 15230 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:54.796281 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.797142 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.799986 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.793598 18635 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.805989 18635 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.807251 15236 raft_consensus.cc:2410] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:54.781684 18556 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:03:54.808168 18635 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.776959 18684 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:54.809150 18684 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:54.809556 18684 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.786491 18517 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:54.810875 18679 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.790s	user 0.000s	sys 0.012s
I20260709 13:03:54.811087 18517 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:54.811647 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:54.774945 18369 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:03:54.812243 15082 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.793998 18538 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:54.813627 15237 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:03:54.813673 18517 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.814147 15237 pending_rounds.cc:85] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 3
I20260709 13:03:54.814427 15237 pending_rounds.cc:107] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:03:54.814698 15237 raft_consensus.cc:2889] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.815734 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.816233 15237 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.818950 15840 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:54.809525 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.825122 15079 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:54.777040 18636 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.833734 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.835525 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.835525 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.838013 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.839164 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:54.841049 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:54.841632 15842 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.841771 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.840046 15845 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:54.837992 15843 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:54.842983 15088 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:54.840432 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:54.844897 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.846287 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:54.848222 15237 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:54.849102 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:54.849722 15054 connection.cc:256] RPC call timeout handler was delayed by -1.0798594330000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
I20260709 13:03:54.850039 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:54.851192 15154 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:54.850668 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 687) took 3029 ms. Trace:
I20260709 13:03:54.860155 15237 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.864012 15082 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:54.850286 15160 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:54.855676 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.857936 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:54.842013 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:54.853700 15230 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:54.865801 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.866657 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:54.859622 15162 rpcz_store.cc:276] 0709 13:03:51.821691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.709897 (+1888206us) service_pool.cc:224] Handling call
0709 13:03:53.919948 (+210051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.920149 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:53.952871 (+ 32722us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.020162 (+ 67291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.020215 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.020261 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.020329 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.850494 (+830165us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540017dc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.850613 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17805,"prepare.queue_time_us":36746,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":6913,"raft.queue_time_us":49156,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4693,"spinlock_wait_cycles":392123776,"thread_start_us":65850,"threads_started":1,"wal-append.queue_time_us":183319}
I20260709 13:03:54.865931 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.873593 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:54.874531 15844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 911) took 1775 ms (client timeout 1621 ms). Trace:
W20260709 13:03:54.874929 15844 rpcz_store.cc:269] 0709 13:03:53.099501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.120322 (+ 20821us) service_pool.cc:224] Handling call
0709 13:03:54.874374 (+1754052us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54006d7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.874468 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496465792}
I20260709 13:03:54.877689 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:54.879365 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55876us
I20260709 13:03:54.884222 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:54.886662 15128 leader_election.cc:400] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:54.887434 15128 leader_election.cc:403] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:03:54.887085 15845 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:54.890514 15231 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:54.890836 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.883162 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.892450 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:54.893783 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53411.3us
I20260709 13:03:54.895978 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:56640 (request call id 16) took 2866 ms. Trace:
I20260709 13:03:54.910385 18622 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.889081 18327 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:54.906293 18517 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:54.896472 15083 rpcz_store.cc:276] 0709 13:03:52.012016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.479354 (+2467338us) service_pool.cc:224] Handling call
0709 13:03:54.877857 (+398503us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9ea735 00007f0d449aad29 00007f0d3df291d4 00007f0d3df29162 00007f0d3df29091 00007f0d3df2900c 00007f0d3df282c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:54.878006 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196727168}
I20260709 13:03:54.925922 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:54.928772 18708 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:54.929174 15230 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:54.928999 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.955848 15162 raft_consensus.cc:2368] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:54.929981 18439 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.930868 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.931803 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:54.932657 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.933964 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:54.934593 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48052.8us
I20260709 13:03:54.934796 18473 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8.
I20260709 13:03:54.851548 18330 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:54.885545 18538 consensus_queue.cc:1243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:54.833211 18636 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:54.928808 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:54.940944 15237 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:54.942210 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2000) took 1803 ms. Trace:
W20260709 13:03:54.956364 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 78747.5us
I20260709 13:03:54.961755 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:54.958403 15159 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
I20260709 13:03:54.964735 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:54.911767 18520 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:55.044499 18520 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:55.045121 18520 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:55.046532 18636 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:54.957851 18629 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:55.048169 18590 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:54.967113 18439 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.048799 18590 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.050876 18590 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:55.050951 18439 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:54.965543 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:54.966832 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.052402 15156 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:03:54.966050 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 147108us
I20260709 13:03:55.054517 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.937456 18598 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:55.040751 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2551) took 3840 ms. Trace:
I20260709 13:03:55.001801 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.060758 15154 rpcz_store.cc:276] 0709 13:03:51.200724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.085730 (+1885006us) service_pool.cc:224] Handling call
0709 13:03:53.086016 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.859533 (+1773517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.040515 (+180982us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:55.040698 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4058892672}
I20260709 13:03:55.041332 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.021746 15160 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:55.005607 15842 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:54.964234 18681 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:54.954627 18556 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.966636 18327 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:54.966609 18616 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.963831 18618 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:55.074560 18584 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:54.958057 18611 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:55.075018 18584 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:55.075384 18611 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:55.075572 18584 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:54.937683 18633 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.076380 18611 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:55.077389 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 111993us
I20260709 13:03:55.078696 18633 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:55.014644 15034 auto_rebalancer.cc:629] Failed to schedule move for tablet 41c6ed4c81c041d18f143b6cd51bea35: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:55.008507 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:55.079895 15162 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:55.040302 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78635.8us
I20260709 13:03:55.065268 15234 rpcz_store.cc:276] 0709 13:03:53.138362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.285122 (+146760us) service_pool.cc:224] Handling call
0709 13:03:53.285543 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.718930 (+1433387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.909322 (+190392us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400318b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:54.942137 (+ 32815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3291899776}
W20260709 13:03:55.087199 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 204101us
I20260709 13:03:55.066529 15232 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:55.090361 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.066790 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:55.096163 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:54.955463 18582 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:55.073060 18549 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:55.072494 18641 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:55.125013 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 235745us
I20260709 13:03:55.125864 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.126113 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:55.065092 18473 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:55.129014 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:55.086594 18629 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:55.064556 18600 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.093833 18439 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:55.119616 18559 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:55.167691 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 283528us
I20260709 13:03:55.177608 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:55.083814 18556 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:55.199229 14984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 355) took 17660 ms (client timeout 15000 ms). Trace:
I20260709 13:03:55.118244 18616 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.160043 18471 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.084065 18681 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.154392 18636 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:55.200006 18327 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.163318 18582 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.124409 18627 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:03:55.225193 18717 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:55.198194 18538 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.168339 18669 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 13:03:55.210237 14984 rpcz_store.cc:269] 0709 13:03:37.539018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.539251 (+   233us) service_pool.cc:224] Handling call
0709 13:03:38.156192 (+616941us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400002490. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8c0640 00007f0d45797b76 00007f0d458a1c3e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:55.199124 (+17042932us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:49.453017 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:49.568657 (+115640us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:49.568728 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:03:49.568757 (+    29us) tablet.cc:662] Decoding operations
0709 13:03:49.666458 (+ 97701us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:49.705488 (+ 39030us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:49.710452 (+  4964us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:03:50.069991 (+359539us) tablet.cc:701] Row locks acquired
0709 13:03:50.070059 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:03:50.070288 (+   229us) write_op.cc:270] Start()
0709 13:03:50.070493 (+   205us) write_op.cc:276] Timestamp: P: 1783602230070218 usec, L: 0
0709 13:03:50.070522 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:03:50.198286 (+127764us) log.cc:844] Serialized 7137 byte log entry
0709 13:03:51.099385 (+901099us) op_driver.cc:464] REPLICATION: finished
0709 13:03:51.589238 (+489853us) write_op.cc:301] APPLY: starting
0709 13:03:51.589351 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:51.589849 (+   498us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:51.604364 (+ 14515us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:51.781697 (+177333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:51.782088 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:03:51.782135 (+    47us) write_op.cc:312] APPLY: finished
0709 13:03:51.782582 (+   447us) log.cc:844] Serialized 125 byte log entry
0709 13:03:51.782777 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:51.782955 (+   178us) write_op.cc:454] Released schema lock
0709 13:03:51.783029 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:51.783482 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:51.783511 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":429694,"reported_tablets":31,"spinlock_wait_cycles":41867648,"child_traces":[["op",{"apply.queue_time_us":460742,"apply.run_cpu_time_us":6418,"apply.run_wall_time_us":194236,"num_ops":18,"prepare.queue_time_us":254151,"prepare.run_cpu_time_us":37588,"prepare.run_wall_time_us":911825,"raft.queue_time_us":163492,"raft.run_cpu_time_us":32260,"raft.run_wall_time_us":216903,"replication_time_us":988920,"thread_start_us":413772,"threads_started":4,"wal-append.queue_time_us":593267,"wal-append.run_cpu_time_us":44869,"wal-append.run_wall_time_us":1980091}]]}
I20260709 13:03:55.169762 18600 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.174190 18621 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:55.262645 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.244781 18635 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:55.245461 18264 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.233229 18633 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:03:55.253047 18719 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:55.257505 18631 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:55.279582 15840 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:55.284372 15234 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:55.284535 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:55.286172 15842 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:55.270306 18717 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:55.254633 18582 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.251600 18636 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:55.312067 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.318861 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 230690us
I20260709 13:03:55.263616 18684 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.268677 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:55.339437 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 70799.4us
I20260709 13:03:55.340423 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.256757 18629 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.350759 15162 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:55.357721 18681 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:55.359375 18616 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:55.295179 18719 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.363641 18520 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.307423 18549 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:03:55.365437 18549 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:55.365724 18719 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.365861 18549 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:55.279924 18264 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.366900 18549 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.368072 18264 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:55.336694 18718 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:03:55.370280 15841 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:55.372546 18660 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.373234 18660 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.319410 18631 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:03:55.376791 14981 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 8, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.377703 18631 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.379381 18556 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.380038 18556 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.380649 18556 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:55.381011 18556 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:55.381953 18600 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:55.381915 18556 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.383157 18556 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:55.383874 18582 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.384487 18582 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:55.340333 18717 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:55.392760 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 314096us
I20260709 13:03:55.395768 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:03:55.401232 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 315699us
I20260709 13:03:55.401677 15237 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:55.402343 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.395192 14981 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:55.417645 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.418085 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.419346 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:55.418470 15235 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:55.385747 18621 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:55.387344 18361 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.386122 18636 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:55.390877 18684 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.384802 18660 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:55.368111 18549 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:55.469555 14981 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 7, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.451871 18473 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:55.454116 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 327956us
I20260709 13:03:55.405815 18717 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:55.486514 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 84262.9us
I20260709 13:03:55.488692 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:55.470082 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57317.1us
I20260709 13:03:55.491736 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.491627 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:55.494112 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 365243us
I20260709 13:03:55.495092 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.454566 18330 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:55.500981 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.360699 18584 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.498885 15814 leader_election.cc:400] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:55.485653 18676 raft_consensus.cc:2764] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:03:55.504078 15814 leader_election.cc:403] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:55.417685 18641 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:03:55.498704 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:55.406021 18582 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.493752 18717 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.510406 14981 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 7, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.495934 18681 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:55.529968 15234 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:55.531961 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:03:55.500326 18684 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:55.559036 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60336.5us
W20260709 13:03:55.509552 18465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.057s	sys 0.055s
I20260709 13:03:55.559880 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.560325 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.562765 14981 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.564023 15155 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:03:55.561996 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:55.565258 15160 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:55.565789 15840 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:55.567205 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:55.568293 15845 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:55.564447 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:55.573725 15841 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:55.575938 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.576594 14981 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 7, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.578418 15846 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:55.528297 18600 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.581859 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.582958 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.529199 18721 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:55.556804 18676 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.554900 18681 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:55.572290 18627 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.594401 14981 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 5, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:03:55.595607 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:55.558079 18695 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:55.596786 15845 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 5: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:55.586449 18471 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:55.557176 18644 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 13:03:55.606496 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.592423 18681 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:55.590437 18676 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:55.608793 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 431215us
I20260709 13:03:55.609853 14981 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 3 to 4, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:55.595134 18627 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.590610 18725 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:55.581833 18455 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:03:55.630546 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 310432us
I20260709 13:03:55.640763 14981 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 5, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 5 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:03:55.639897 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.010s
I20260709 13:03:55.596674 18520 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:55.646608 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.645534 14981 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 7 to 8, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.590243 18641 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:03:55.606081 18730 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:55.647756 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.011s	sys 0.010s
W20260709 13:03:55.694913 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 568890us
W20260709 13:03:55.695513 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.602519 18622 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9.
I20260709 13:03:55.600555 18659 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:55.638962 18627 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.588119 18559 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:03:55.691401 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 351055us
I20260709 13:03:55.709064 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.607076 18471 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:55.608224 18721 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:55.697618 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 192290us
I20260709 13:03:55.635540 18676 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.648762 18725 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.582171 18473 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.718354 14981 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 7, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.646447 18636 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.659817 18660 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.659196 18732 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.697180 18730 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:55.700276 18520 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.756989 18732 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:55.757973 18729 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:55.758958 18378 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:55.739158 18725 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:55.766204 14981 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 6, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.777309 15841 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:55.734783 18471 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:55.787366 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 227560us
I20260709 13:03:55.798139 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:55.802666 14981 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.819895 14981 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 6, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.820550 18745 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:55.786538 18340 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:55.768208 18721 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:03:55.793177 18378 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:55.805998 18726 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:03:55.775539 18636 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.815964 18465 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:55.788790 18520 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:55.848397 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.873519 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 931) took 1115 ms. Trace:
W20260709 13:03:55.846505 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 322068us
I20260709 13:03:55.855068 14981 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 5, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 5 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:55.860386 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 163491us
W20260709 13:03:55.862529 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 152432us
I20260709 13:03:55.845398 18734 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.875778 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:55.874022 15847 rpcz_store.cc:276] 0709 13:03:54.757724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.757831 (+   107us) service_pool.cc:224] Handling call
0709 13:03:54.758150 (+   319us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:54.763298 (+  5148us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:55.019302 (+256004us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b18003e8720. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:03:55.260830 (+241528us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.777028 (+516198us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.777630 (+   602us) log.cc:844] Serialized 52 byte log entry
0709 13:03:55.777804 (+   174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:55.777855 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.782017 (+  4162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.873127 (+ 91110us) raft_consensus.cc:1685] finished
0709 13:03:55.873238 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.873448 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13805,"prepare.queue_time_us":546546,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":2551,"raft.queue_time_us":132005,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":11274,"spinlock_wait_cycles":17747840,"thread_start_us":723247,"threads_started":3,"wal-append.queue_time_us":471145}
I20260709 13:03:55.854131 18582 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:55.880246 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.877311 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.882318 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.816322 18361 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:55.876714 15128 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:55.898540 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50207.3us
I20260709 13:03:55.895349 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.896288 14981 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 8 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:03:55.862075 18465 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:55.905990 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:55.874259 18737 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.913933 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:55.912961 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.911792 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:55.922665 15155 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:55.922888 14981 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 1 to 3, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 3 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:03:55.932073 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1844) took 1142 ms. Trace:
I20260709 13:03:55.933813 15162 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:55.932760 15233 rpcz_store.cc:276] 0709 13:03:54.789990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.886186 (+ 96196us) service_pool.cc:224] Handling call
0709 13:03:54.886506 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.886888 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:54.886919 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.957766 (+ 70847us) log.cc:844] Serialized 26 byte log entry
0709 13:03:55.243120 (+285354us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:55.243180 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.243297 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.931766 (+688469us) raft_consensus.cc:1685] finished
0709 13:03:55.931799 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.931991 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":291,"raft.run_cpu_time_us":2746,"raft.run_wall_time_us":53555,"thread_start_us":207142,"threads_started":1,"wal-append.queue_time_us":412556}
I20260709 13:03:55.917680 18473 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:55.917074 18669 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:55.842159 18730 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:55.967942 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90704.1us
I20260709 13:03:55.969108 14981 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 5, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 5 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.974967 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:55.918464 18582 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:03:55.916730 18734 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:03:55.951388 18745 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.991890 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:55.958905 18669 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:55.951256 18759 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.963629 18473 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:56.021615 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:56.023001 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 186027us
I20260709 13:03:56.035329 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.039958 15079 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:56.000581 18611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.013s	sys 0.013s
I20260709 13:03:55.993739 18755 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:55.977408 18760 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.960784 18378 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.087764 18644 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.088519 18644 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.091378 18644 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.092523 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:56.093480 15847 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:03:56.098451 18556 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.002s
I20260709 13:03:56.099161 18556 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:56.048907 18631 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.107134 18631 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:55.931422 18737 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.109874 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.067735 18760 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:56.112514 18760 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:56.021776 18669 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:56.059877 18611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.013s	sys 0.013s
I20260709 13:03:56.034442 18721 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:56.161567 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.070192 18732 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:56.155148 18659 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.161137 18340 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.177542 15079 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:56.177999 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.193135 15083 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:56.193406 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 317683us
I20260709 13:03:56.192268 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.211441 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.212185 15079 raft_consensus.cc:2368] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7.
I20260709 13:03:56.021063 18745 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:56.145460 18627 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.220083 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.224361 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:56.220748 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.221076 15087 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:56.217666 15086 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:56.192317 18582 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:56.226660 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:56.228899 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.176707 18520 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:56.234477 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:56.236176 15079 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:56.213317 18340 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.269311 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.278365 15161 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.286286 18631 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:56.286417 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48746.6us
I20260709 13:03:56.234184 18471 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.294307 18778 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:56.296108 18717 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.734s	user 0.006s	sys 0.006s
W20260709 13:03:56.297652 18764 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:03:56.299355 18747 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.287500 18644 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:03:56.233290 18758 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:03:56.311192 18743 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:03:56.233108 18752 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:03:56.233250 18710 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:03:56.233186 18741 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
I20260709 13:03:56.311672 18465 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:56.232975 18757 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:03:56.335549 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:56.209982 18659 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:56.420168 15840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.028s	sys 0.025s
I20260709 13:03:56.244431 18755 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:56.482326 15840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.028s	sys 0.025s
I20260709 13:03:56.292567 18734 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:56.436676 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:56.438836 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.210544 18473 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:56.403766 18361 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:56.467953 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 132535us
I20260709 13:03:56.548034 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:56.289196 18582 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.492555 15081 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:56.549952 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.438411 18737 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.454346 18778 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:56.566186 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:56.299000 18771 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.323s
W20260709 13:03:56.553190 18758 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:03:56.423377 18717 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:56.630533 18763 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.486s	user 0.000s	sys 0.001s
W20260709 13:03:56.641480 18741 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:03:56.230723 18759 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.419s
W20260709 13:03:56.647054 18757 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:03:56.557035 18773 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:56.685848 18676 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.944s	user 0.010s	sys 0.002s
I20260709 13:03:56.433245 18747 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:56.719761 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:56.699810 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.030s	sys 0.023s
W20260709 13:03:56.736159 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.030s	sys 0.031s
W20260709 13:03:56.647212 18710 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:03:56.728221 15848 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:56.730146 15842 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:56.730682 15087 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:56.736745 15083 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 13:03:56.564808 18361 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.757391 18361 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.755218 15083 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:56.758960 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.759753 15083 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:56.760217 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 944) took 1824 ms. Trace:
I20260709 13:03:56.760457 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.761523 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.760591 15843 rpcz_store.cc:276] 0709 13:03:54.935487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.935617 (+   130us) service_pool.cc:224] Handling call
0709 13:03:54.935901 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.982550 (+ 46649us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:55.277987 (+295437us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.846775 (+568788us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.889175 (+ 42400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:55.889240 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:55.889415 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.760025 (+870610us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b54005fd554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:56.760066 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.760173 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97587,"prepare.queue_time_us":396076,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":237758,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":6265,"spinlock_wait_cycles":113214464,"thread_start_us":776654,"threads_started":3,"wal-append.queue_time_us":454296}
I20260709 13:03:56.761996 15083 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:56.764247 18469 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.766696 18469 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.768685 18469 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.769670 15083 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:03:56.770633 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.771199 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:56.772018 15083 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:56.773155 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:56.773618 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 741) took 1479 ms. Trace:
I20260709 13:03:56.773999 15234 rpcz_store.cc:276] 0709 13:03:55.294638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.295261 (+   623us) service_pool.cc:224] Handling call
0709 13:03:55.295602 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.555918 (+260316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.773400 (+1217482us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002d4954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:56.773562 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538550144}
I20260709 13:03:56.776661 18264 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:03:56.777974 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.777680 18264 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:56.786574 18779 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:03:56.788046 18590 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:56.788612 18590 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:56.794111 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:56.723398 18549 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.232s	user 0.013s	sys 0.008s
I20260709 13:03:56.820569 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:56.735424 18760 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.622s	user 0.004s	sys 0.008s
I20260709 13:03:56.824714 15233 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:56.825553 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:56.826395 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:56.832599 15228 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:56.731360 18782 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.831808 15230 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:56.840130 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:56.841099 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:56.852622 15236 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:03:56.834460 18757 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:56.831116 18361 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.887804 18340 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:56.889195 18659 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:56.805372 18736 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:56.842505 18770 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.619s	user 0.001s	sys 0.000s
I20260709 13:03:56.821015 18520 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.831624 18631 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:56.753496 18676 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:56.899873 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.845832 18627 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:56.920667 15080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.034s	sys 0.075s
I20260709 13:03:56.872056 18264 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:03:56.928053 15080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.035s	sys 0.075s
W20260709 13:03:56.931061 15126 connection.cc:256] RPC call timeout handler was delayed by -1.4380507000000762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 13:03:56.789690 18590 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.892982 18745 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:56.838327 18760 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.963912 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:56.965417 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:56.973611 15159 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:56.974305 15083 raft_consensus.cc:2368] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:03:56.976229 15156 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:56.979133 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:56.982256 15154 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:56.928813 18471 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.868058 18782 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:56.983875 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:56.932507 18721 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:56.987787 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:56.894737 18361 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:56.990572 15159 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:56.932585 18631 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.004518 15154 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:56.977306 18745 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:56.981516 18590 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:56.928474 18520 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:57.018889 18785 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:03:57.019939 15231 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:03:57.035282 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2026) took 1151 ms. Trace:
I20260709 13:03:57.035647 15231 rpcz_store.cc:276] 0709 13:03:55.883785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.883913 (+   128us) service_pool.cc:224] Handling call
0709 13:03:55.884222 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.019573 (+1135351us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b54001fc0a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:03:57.035010 (+ 15437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.035224 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47323904}
I20260709 13:03:56.919570 18736 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.021262 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2040) took 1459 ms. Trace:
W20260709 13:03:57.025622 18783 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:03:57.025439 18695 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:57.027776 18636 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.027904 18631 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.032485 18582 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:56.903232 18778 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.003994 18773 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.047957 18784 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.340s	user 0.001s	sys 0.001s
I20260709 13:03:57.175803 18750 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:57.109401 18790 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:03:57.172989 18773 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.183214 18760 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:57.161053 18791 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:03:57.149418 18789 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:03:57.176635 18750 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:57.084635 18736 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.194476 18782 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:57.229334 14981 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.013s	sys 0.002s
W20260709 13:03:57.229930 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 407637us
W20260709 13:03:57.245455 15082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.260s	user 0.008s	sys 0.022s
W20260709 13:03:57.246183 15082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.261s	user 0.008s	sys 0.022s
I20260709 13:03:57.248857 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1848) took 1034 ms. Trace:
I20260709 13:03:57.249614 15085 rpcz_store.cc:276] 0709 13:03:56.214608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.214726 (+   118us) service_pool.cc:224] Handling call
0709 13:03:56.215015 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.215230 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:56.551747 (+336517us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.246969 (+695222us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.247692 (+   723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.247774 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:57.247949 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.248142 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.248511 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20391,"prepare.queue_time_us":540707,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":182624,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":152,"thread_start_us":1007251,"threads_started":3,"wal-append.queue_time_us":465238}
I20260709 13:03:57.254179 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:57.255999 15085 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:03:57.256700 15088 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.022s
W20260709 13:03:57.273195 15088 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.022s
I20260709 13:03:57.275815 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2457) took 1296 ms. Trace:
W20260709 13:03:57.258136 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.011s	sys 0.026s
W20260709 13:03:57.259362 15158 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.005s	sys 0.018s
W20260709 13:03:57.277235 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.012s	sys 0.026s
W20260709 13:03:57.277315 15158 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.005s	sys 0.019s
I20260709 13:03:57.278911 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 716) took 1368 ms. Trace:
I20260709 13:03:57.037281 15161 rpcz_store.cc:276] 0709 13:03:55.561856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.562651 (+   795us) service_pool.cc:224] Handling call
0709 13:03:55.585569 (+ 22918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.291175 (+705606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.020917 (+729742us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54003e1a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:57.021181 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592691072}
I20260709 13:03:57.263190 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:57.282868 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:57.279337 15158 rpcz_store.cc:276] 0709 13:03:55.910121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.914589 (+  4468us) service_pool.cc:224] Handling call
0709 13:03:55.929419 (+ 14830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.929602 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:55.961452 (+ 31850us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.278058 (+1316606us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.278235 (+   177us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:57.278369 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.278509 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.278737 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35477,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":5611,"raft.queue_time_us":32524,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":4084096,"thread_start_us":1315865,"threads_started":2,"wal-append.queue_time_us":941662}
I20260709 13:03:57.283981 15161 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:03:57.151927 18681 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.507s	user 0.013s	sys 0.000s
I20260709 13:03:57.048125 18717 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:57.341459 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 299785us
I20260709 13:03:57.362674 18520 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:57.363498 15847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.003s	sys 0.024s
W20260709 13:03:57.363979 15847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.003s	sys 0.024s
W20260709 13:03:57.357095 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 122992us
I20260709 13:03:57.366863 18782 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.369489 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 952) took 1583 ms. Trace:
I20260709 13:03:57.369936 15840 rpcz_store.cc:276] 0709 13:03:55.785948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.786166 (+   218us) service_pool.cc:224] Handling call
0709 13:03:55.786623 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.482858 (+696235us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:56.482901 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.483490 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.732127 (+248637us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:56.732163 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.732226 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.368869 (+636643us) raft_consensus.cc:1685] finished
0709 13:03:57.368914 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.369191 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4911,"raft.queue_time_us":101814,"spinlock_wait_cycles":18635136,"thread_start_us":938941,"threads_started":2,"wal-append.queue_time_us":410874}
I20260709 13:03:57.373463 18361 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:57.374269 18361 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:57.375027 18361 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.378369 18750 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:57.380223 18750 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:57.276481 15088 rpcz_store.cc:276] 0709 13:03:55.989995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.990144 (+   149us) service_pool.cc:224] Handling call
0709 13:03:55.990546 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.175759 (+185213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.175799 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.177879 (+  2080us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.640578 (+462699us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.274235 (+633657us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.274946 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.275164 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.275314 (+   150us) raft_consensus.cc:1685] finished
0709 13:03:57.275337 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.275677 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":115595,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":109653,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":1145038,"threads_started":3,"wal-append.queue_time_us":600354}
I20260709 13:03:57.351485 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.353175 18684 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.354250 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.355894 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1852) took 1139 ms. Trace:
I20260709 13:03:57.356194 18747 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:57.075409 18719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.014s	sys 0.013s
I20260709 13:03:57.384828 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:57.385564 15157 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:03:57.386511 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2021) took 1676 ms. Trace:
W20260709 13:03:57.387833 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.387954 18636 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.390084 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.390656 18804 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:57.392422 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:57.393191 15231 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:03:57.368096 18782 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.393461 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:57.394626 18520 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:57.399163 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1001) took 1166 ms. Trace:
I20260709 13:03:57.416266 15231 pending_rounds.cc:85] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 2
I20260709 13:03:57.044121 18695 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.403023 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.405055 15162 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:57.405158 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:57.406361 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:57.407684 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:57.408006 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:57.382913 15087 rpcz_store.cc:276] 0709 13:03:56.215841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.238235 (+ 22394us) service_pool.cc:224] Handling call
0709 13:03:57.355327 (+1117092us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540011a8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:57.355653 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010761088}
W20260709 13:03:57.409013 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.407684 18465 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:57.409216 18378 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:57.423632 15231 pending_rounds.cc:107] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:03:57.423580 15842 rpcz_store.cc:276] 0709 13:03:56.232809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.232969 (+   160us) service_pool.cc:224] Handling call
0709 13:03:57.398594 (+1165625us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:57.398872 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785573248}
I20260709 13:03:57.431298 15231 raft_consensus.cc:2889] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:57.443756 15088 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:57.447820 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 951) took 1637 ms. Trace:
I20260709 13:03:57.449087 15156 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.453646 18800 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.454579 18800 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.456323 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:57.457020 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:57.457242 15848 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:57.457834 15157 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:57.457971 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:57.460111 18636 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.454948 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:57.460690 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.465955 18616 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.466766 18616 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.461349 15840 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
W20260709 13:03:57.468999 18719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.019s	sys 0.023s
I20260709 13:03:57.182217 18676 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:57.475584 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49228.8us
I20260709 13:03:57.478197 18771 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:57.478477 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:57.478879 15081 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.927s	user 0.020s	sys 0.009s
I20260709 13:03:57.479401 15081 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:03:57.479117 18771 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.479954 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2464) took 1254 ms. Trace:
I20260709 13:03:57.480139 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.480388 15081 rpcz_store.cc:276] 0709 13:03:56.225093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.419039 (+193946us) service_pool.cc:224] Handling call
0709 13:03:57.479878 (+1060839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.481097 18804 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.481685 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:57.482316 15081 raft_consensus.cc:2376] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:03:57.482141 18681 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.483387 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.485832 18773 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.486778 18616 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:57.487962 18330 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.484736 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.488965 18330 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.483808 18736 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.491899 15086 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:57.492671 15160 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.493299 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:57.494616 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.494846 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:57.495374 15086 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:57.495898 15229 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:57.497242 18735 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.498059 18520 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.498107 18735 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.501785 15230 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:57.413585 15235 rpcz_store.cc:276] 0709 13:03:55.709799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.710465 (+   666us) service_pool.cc:224] Handling call
0709 13:03:55.744132 (+ 33667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.890736 (+146604us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:55.890797 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.891480 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.057856 (+166376us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:56.057930 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.385845 (+1327915us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400057dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:57.385902 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.385991 (+    89us) raft_consensus.cc:1685] finished
0709 13:03:57.386007 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.386275 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1018,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":18454,"spinlock_wait_cycles":335924992,"thread_start_us":111900,"threads_started":1,"wal-append.queue_time_us":317977,"wal-append.run_cpu_time_us":2187,"wal-append.run_wall_time_us":1107511}
I20260709 13:03:57.505053 18800 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:57.506338 18730 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:57.487759 18684 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.508996 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:57.510234 15163 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:57.511826 15163 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:57.496452 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.492259 18736 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.527978 15236 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:57.541654 15848 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:03:57.541944 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.542829 18800 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:57.543583 18800 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:57.544553 18669 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.377s	user 0.010s	sys 0.018s
I20260709 13:03:57.545468 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:57.546502 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:57.550459 15156 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:57.545384 18616 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:57.545367 15086 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:57.556392 15232 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:57.448342 15844 rpcz_store.cc:276] 0709 13:03:55.763753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.769753 (+  6000us) service_pool.cc:224] Handling call
0709 13:03:55.770717 (+   964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.934168 (+163451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.934211 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.934729 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.419493 (+484764us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:56.419553 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.400071 (+980518us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b540036ced4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:57.400194 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.400294 (+   100us) raft_consensus.cc:1685] finished
0709 13:03:57.400321 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.400757 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19396,"raft.queue_time_us":109574,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":24721,"spinlock_wait_cycles":91121408,"thread_start_us":607630,"threads_started":2,"wal-append.queue_time_us":631436}
I20260709 13:03:57.708974 15841 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:58.006153 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:57.742502 15843 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:03:57.679304 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56671.5us
I20260709 13:03:57.621477 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:57.804250 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:57.945714 15160 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:57.678761 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:58.011420 18780 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:58.012074 18780 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:57.992074 18612 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:58.017856 18612 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:58.018787 18814 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.553s	user 0.000s	sys 0.001s
I20260709 13:03:58.020531 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 897) took 1272 ms. Trace:
I20260709 13:03:58.021077 15086 rpcz_store.cc:276] 0709 13:03:56.730117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.496075 (+765958us) service_pool.cc:224] Handling call
0709 13:03:58.002561 (+506486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.022208 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.022652 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.023299 15086 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:58.023862 18465 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.005s
I20260709 13:03:58.023926 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 891) took 1468 ms. Trace:
I20260709 13:03:57.548682 18636 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.025240 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2456) took 2045 ms. Trace:
W20260709 13:03:57.686415 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 99417.4us
W20260709 13:03:57.769932 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 126339us
W20260709 13:03:57.778509 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.024524 15086 rpcz_store.cc:276] 0709 13:03:56.555561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.021805 (+1466244us) service_pool.cc:224] Handling call
0709 13:03:58.023825 (+  2020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.764704 15157 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:03:57.484679 18681 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:57.821441 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1854) took 1604 ms (client timeout 2121 ms). Trace:
I20260709 13:03:57.703806 15162 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:57.979734 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.035043 15230 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.665318 15202 leader_election.cc:400] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7.
I20260709 13:03:58.036589 15202 leader_election.cc:403] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:03:58.036779 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:57.818277 18816 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
W20260709 13:03:57.826233 15814 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.509s (SENT) [suppressed 25 similar messages]
I20260709 13:03:57.524535 18330 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.041018 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.036697 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 995) took 1876 ms. Trace:
I20260709 13:03:58.026176 18669 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.044409 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:57.991649 18771 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.595293 18684 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.644608 18735 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.046348 15236 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:03:58.047693 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 582317us
W20260709 13:03:58.048028 18725 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 2.240s	user 0.011s	sys 0.011s
W20260709 13:03:58.048712 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 411573us
I20260709 13:03:57.712692 18793 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.719542 18788 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.573s	user 0.001s	sys 0.000s
W20260709 13:03:57.781257 18787 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.806s	user 0.004s	sys 0.000s
I20260709 13:03:57.976272 18361 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.075093 18669 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.075338 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1850) took 1859 ms. Trace:
I20260709 13:03:58.050117 15847 rpcz_store.cc:276] 0709 13:03:56.160621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.162179 (+  1558us) service_pool.cc:224] Handling call
0709 13:03:56.162526 (+   347us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:56.162962 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.162990 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:56.165935 (+  2945us) log.cc:844] Serialized 78 byte log entry
0709 13:03:57.418528 (+1252593us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:57.777181 (+358653us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.979706 (+202525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.993285 (+ 13579us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.993379 (+    94us) raft_consensus.cc:1685] finished
0709 13:03:57.993425 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.036639 (+ 43214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119746,"raft.queue_time_us":119787,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":5015,"thread_start_us":1470341,"threads_started":2,"wal-append.queue_time_us":1009786}
I20260709 13:03:58.076943 18744 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:58.076011 15080 rpcz_store.cc:276] 0709 13:03:56.215190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.218242 (+  3052us) service_pool.cc:224] Handling call
0709 13:03:56.218662 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.218945 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.218982 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.219543 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.928659 (+709116us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:56.928712 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.928812 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.718539 (+789727us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:03:58.074592 (+356053us) raft_consensus.cc:1685] finished
0709 13:03:58.074632 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.075027 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312576,"thread_start_us":523890,"threads_started":1,"wal-append.queue_time_us":822903}
I20260709 13:03:58.056196 15156 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:58.080083 15080 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:57.697806 18804 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:57.816509 18695 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.082191 18695 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:58.083737 18750 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.703s	user 0.014s	sys 0.000s
I20260709 13:03:57.947359 18520 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.045776 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2472) took 1280 ms. Trace:
W20260709 13:03:58.034555 15083 rpcz_store.cc:269] 0709 13:03:56.216437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.772867 (+556430us) service_pool.cc:224] Handling call
0709 13:03:57.820953 (+1048086us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400191c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:57.821218 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460456320}
I20260709 13:03:58.084530 15088 rpcz_store.cc:276] 0709 13:03:56.765474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.803686 (+1038212us) service_pool.cc:224] Handling call
0709 13:03:58.045677 (+241991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.712582 18772 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:58.085330 15081 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:58.041951 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.025766 15082 rpcz_store.cc:276] 0709 13:03:55.979382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.979522 (+   140us) service_pool.cc:224] Handling call
0709 13:03:55.984942 (+  5420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.247132 (+1262190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.247237 (+   105us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.248629 (+  1392us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.339060 (+ 90431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.391365 (+ 52305us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.481049 (+ 89684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.481167 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.024932 (+543765us) raft_consensus.cc:1685] finished
0709 13:03:58.024959 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.025180 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127124,"prepare.queue_time_us":54902,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":205507,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":211,"thread_start_us":1350966,"threads_started":2,"wal-append.queue_time_us":90324}
I20260709 13:03:58.063781 15087 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:58.086612 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2468) took 1605 ms. Trace:
I20260709 13:03:58.052429 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.064755 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:58.066227 18465 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:58.087144 15087 rpcz_store.cc:276] 0709 13:03:56.481410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.666004 (+1184594us) service_pool.cc:224] Handling call
0709 13:03:58.086539 (+420535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.077780 18744 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:57.707679 18611 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.054064 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:58.064361 18718 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7.
I20260709 13:03:58.088140 15160 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:58.059924 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.093210 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:58.094723 18815 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.569s	user 0.000s	sys 0.001s
I20260709 13:03:58.089450 18772 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:58.084832 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.112150 15159 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:57.781181 18721 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.332s
I20260709 13:03:58.100684 18793 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:58.122148 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 92242.1us
W20260709 13:03:58.125847 18778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.023s	sys 0.030s
I20260709 13:03:58.123209 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:58.112797 18782 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:58.133620 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:58.108747 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1853) took 1872 ms. Trace:
I20260709 13:03:58.142897 15080 rpcz_store.cc:276] 0709 13:03:56.216154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.079016 (+1862862us) service_pool.cc:224] Handling call
0709 13:03:58.079659 (+   643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.080767 (+  1108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.088475 (+  7708us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:58.136252 18778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.025s	sys 0.030s
I20260709 13:03:58.124238 18736 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.154739 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.088528 18695 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:58.154196 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:58.136155 18792 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:58.098212 18725 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:58.166185 15232 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:58.123855 18616 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.178838 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:58.179195 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.179736 15162 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:58.179831 15846 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:58.187557 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.190912 18473 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.191803 18473 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.181203 18584 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.163122 18520 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:58.164202 18792 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.198856 15236 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.190912 18465 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:58.206388 18792 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:58.190160 18730 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.003s
I20260709 13:03:58.173012 18800 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.183981 18782 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:58.209681 18465 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:58.186187 18778 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:58.217073 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:58.218540 15088 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:03:58.137087 18611 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.233431 15084 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:58.235586 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2465) took 1981 ms. Trace:
I20260709 13:03:58.236145 15088 rpcz_store.cc:276] 0709 13:03:56.249728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.085377 (+1835649us) service_pool.cc:224] Handling call
0709 13:03:58.163784 (+ 78407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.231151 (+ 67367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.231503 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.241307 15839 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:03:58.241745 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1857) took 1418 ms. Trace:
I20260709 13:03:58.227988 18520 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:03:58.251772 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 197978us
I20260709 13:03:58.253141 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:58.250499 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:58.262493 15088 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:03:58.269114 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 203468us
I20260709 13:03:58.247058 15234 rpcz_store.cc:276] 0709 13:03:56.823088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.825131 (+  2043us) service_pool.cc:224] Handling call
0709 13:03:56.825471 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.825717 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:56.835189 (+  9472us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.824588 (+989399us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b18002c4160. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:03:58.234823 (+410235us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.241180 (+  6357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.241255 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:58.241327 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.241422 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.241594 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1533,"prepare.queue_time_us":13701,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2392,"raft.queue_time_us":6970,"spinlock_wait_cycles":8370944,"thread_start_us":394338,"threads_started":1}
I20260709 13:03:58.313048 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2475) took 1504 ms. Trace:
I20260709 13:03:58.313498 15088 rpcz_store.cc:276] 0709 13:03:56.766307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.242518 (+1476211us) service_pool.cc:224] Handling call
0709 13:03:58.261953 (+ 19435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.265315 (+  3362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.270646 (+  5331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.319432 15232 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 5.
W20260709 13:03:58.332451 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 243678us
I20260709 13:03:58.312089 15236 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:03:58.337222 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.314911 18473 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.246385 18772 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.266258 18695 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.270542 18778 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.367720 18616 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:58.384446 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 182720us
I20260709 13:03:58.368594 18744 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:58.391141 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.380911 18725 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:58.421355 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 595111us
W20260709 13:03:58.421625 15229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.024s	sys 0.028s
W20260709 13:03:58.422111 15229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.024s	sys 0.029s
W20260709 13:03:58.388741 18801 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.036s	sys 0.031s
I20260709 13:03:58.427211 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:58.427394 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 108140us
I20260709 13:03:58.410008 18616 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:58.375336 18465 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.438990 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.443176 15843 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:58.436488 18611 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:58.449671 18817 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
W20260709 13:03:58.450117 18825 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:03:58.427955 18520 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.451630 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2044) took 1164 ms. Trace:
W20260709 13:03:58.451961 18819 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:03:58.452070 15154 rpcz_store.cc:276] 0709 13:03:57.286870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.349207 (+ 62337us) service_pool.cc:224] Handling call
0709 13:03:57.350051 (+   844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.358633 (+  8582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.358673 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.359367 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.610385 (+251018us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:57.610433 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.610519 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.451304 (+840785us) raft_consensus.cc:1685] finished
0709 13:03:58.451337 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.451466 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7073,"raft.queue_time_us":8653,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":389,"thread_start_us":250464,"threads_started":1,"wal-append.queue_time_us":777383}
I20260709 13:03:58.453527 18826 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:58.454617 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2022) took 2744 ms. Trace:
I20260709 13:03:58.454313 18826 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.455018 15237 rpcz_store.cc:276] 0709 13:03:55.710075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.710199 (+   124us) service_pool.cc:224] Handling call
0709 13:03:55.710572 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.718029 (+  7457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.718071 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.825607 (+2107536us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b2c00032e28. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:03:57.826369 (+   762us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.200850 (+374481us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:58.204586 (+  3736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.206942 (+  2356us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.454129 (+247187us) raft_consensus.cc:1685] finished
0709 13:03:58.454148 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.454409 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7033,"raft.queue_time_us":52025,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":36134272,"thread_start_us":367659,"threads_started":1,"wal-append.queue_time_us":495800}
W20260709 13:03:58.448216 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 425828us
W20260709 13:03:58.469239 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.432644 18750 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:58.472733 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.446663 18473 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:58.510062 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 191828us
I20260709 13:03:58.510604 18780 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:03:58.512962 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 86145.6us
I20260709 13:03:58.515393 18721 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:58.518028 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.518164 18721 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:58.518618 18721 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:58.518750 15159 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:58.519861 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:58.519693 18721 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:58.520445 15159 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:58.520967 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.521095 18721 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:58.521683 15160 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:58.522547 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.523141 15160 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:58.523904 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.525234 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.525242 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.525653 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.526376 15156 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:58.526742 15155 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.527308 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.483179 18782 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.529117 15160 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:58.534091 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.544888 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.545789 15846 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:03:58.549422 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:58.551014 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45754.7us
I20260709 13:03:58.552883 15159 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:58.555449 18835 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:58.556190 18835 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.556689 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.557639 15848 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:58.558077 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1006) took 1014 ms. Trace:
I20260709 13:03:58.558668 18835 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.529330 18782 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.560947 18793 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.548357 18772 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:58.562930 18782 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:58.563844 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.558745 18695 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.561151 15154 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:58.569715 15843 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:58.558437 15848 rpcz_store.cc:276] 0709 13:03:57.543411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.583039 (+ 39628us) service_pool.cc:224] Handling call
0709 13:03:58.556562 (+973523us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:58.558018 (+  1456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843030272}
I20260709 13:03:58.499683 18611 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:58.605284 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 303957us
I20260709 13:03:58.485396 18616 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.621299 15848 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:03:58.629442 15230 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.001s
I20260709 13:03:58.631189 15230 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:58.576936 18792 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.635046 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1855) took 2418 ms. Trace:
I20260709 13:03:58.633810 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2476) took 1862 ms. Trace:
W20260709 13:03:58.636353 15084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2462) took 2463 ms (client timeout 2509 ms). Trace:
W20260709 13:03:58.703213 15084 rpcz_store.cc:269] 0709 13:03:56.172770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.172940 (+   170us) service_pool.cc:224] Handling call
0709 13:03:58.636062 (+2463122us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b540039b554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:58.636249 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4610354048}
I20260709 13:03:58.704957 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.706883 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.708356 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:58.638381 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.706923 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.642773 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.631902 18744 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:58.645305 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.646255 15086 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:58.648161 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.647322 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.650492 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:58.651705 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.651830 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:58.658967 18737 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:58.660697 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 955) took 2224 ms (client timeout 1546 ms). Trace:
I20260709 13:03:58.664384 18718 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:58.665587 15810 connection.cc:256] RPC call timeout handler was delayed by -1.0439170059999014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 13:03:58.666265 18736 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:58.668488 15085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 890) took 2113 ms (client timeout 2091 ms). Trace:
W20260709 13:03:58.725751 15085 rpcz_store.cc:269] 0709 13:03:56.555255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.668206 (+2112951us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:58.668388 (+   182us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:58.674986 18612 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.683741 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 994) took 2887 ms. Trace:
I20260709 13:03:58.700898 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2034) took 1104 ms. Trace:
W20260709 13:03:58.701097 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 137471us
W20260709 13:03:58.639184 15081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2470) took 1954 ms (client timeout 2045 ms). Trace:
I20260709 13:03:58.701714 15083 rpcz_store.cc:276] 0709 13:03:56.216737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.084864 (+1868127us) service_pool.cc:224] Handling call
0709 13:03:58.093355 (+  8491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.163082 (+ 69727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.163132 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.177626 (+ 14494us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:58.194342 (+ 16716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.634751 (+440409us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:58.634811 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.634979 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95259904}
W20260709 13:03:58.644170 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1846) took 2430 ms (client timeout 3159 ms). Trace:
W20260709 13:03:58.719195 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90864.5us
W20260709 13:03:58.717891 15839 rpcz_store.cc:269] 0709 13:03:56.436182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.436368 (+   186us) service_pool.cc:224] Handling call
0709 13:03:58.660389 (+2224021us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:58.660580 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3998237568}
I20260709 13:03:58.593308 18473 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:58.729278 15081 rpcz_store.cc:269] 0709 13:03:56.685294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.484377 (+799083us) service_pool.cc:224] Handling call
0709 13:03:58.638956 (+1154579us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:58.639116 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367482624}
I20260709 13:03:58.725301 15157 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:58.733389 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1007) took 1189 ms. Trace:
I20260709 13:03:58.733881 15840 rpcz_store.cc:276] 0709 13:03:57.543806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.584569 (+ 40763us) service_pool.cc:224] Handling call
0709 13:03:58.645136 (+1060567us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:58.733316 (+ 88180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6265600}
I20260709 13:03:58.699628 18684 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:58.728281 15230 rpcz_store.cc:276] 0709 13:03:57.526891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.528970 (+  2079us) service_pool.cc:224] Handling call
0709 13:03:58.631554 (+1102584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.728174 15841 rpcz_store.cc:276] 0709 13:03:55.796317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.796525 (+   208us) service_pool.cc:224] Handling call
0709 13:03:55.796931 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.045116 (+2248185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.683344 (+638228us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400771d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:58.683646 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3888063232}
I20260709 13:03:58.700999 18826 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:58.729996 15079 rpcz_store.cc:269] 0709 13:03:56.213843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.219852 (+  6009us) service_pool.cc:224] Handling call
0709 13:03:58.643798 (+2423946us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5400349254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:58.644010 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26953472}
I20260709 13:03:58.720642 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:03:58.740475 15842 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.733273 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2466) took 2311 ms. Trace:
W20260709 13:03:58.724295 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72742.8us
I20260709 13:03:58.741336 15080 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.720021 18725 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.745828 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.752981 15154 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:03:58.753070 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.754190 15081 raft_consensus.cc:2376] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9. Re-sending same reply.
I20260709 13:03:58.754037 15085 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:03:58.729878 18778 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.700723 18611 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.759529 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.736920 18684 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:03:58.772300 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.775079 18800 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:58.775683 18800 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.775805 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.776165 18800 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.777143 15085 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:58.778293 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.780113 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.781227 15841 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:58.781991 18840 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.782740 18840 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:58.785568 18832 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:58.787675 18848 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:03:58.790009 18792 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.767820 18718 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:58.793591 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.794534 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2471) took 2061 ms. Trace:
I20260709 13:03:58.794404 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.795047 15086 rpcz_store.cc:276] 0709 13:03:56.733502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.116050 (+1382548us) service_pool.cc:224] Handling call
0709 13:03:58.645754 (+529704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.715907 (+ 70153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.794318 (+ 78411us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400349250. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:58.794473 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128787328}
I20260709 13:03:58.796393 18684 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:58.797817 18684 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:58.799451 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.799937 15848 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:58.800100 15840 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:03:58.767570 18782 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:58.801050 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.801489 18782 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.802069 18782 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.804126 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.805549 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.806172 15079 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:58.807138 18827 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:58.807154 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.796473 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.807934 15079 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.808782 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.794464 15080 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:03:58.809389 15079 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:58.809401 15086 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:58.809962 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.811138 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.805703 18744 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:58.812331 18744 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.813020 18835 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.813409 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.813666 18835 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.814244 15081 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:58.815623 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.816169 18835 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.816978 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:58.818241 18473 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.819054 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.819849 15086 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.790035 18734 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:03:58.825731 18845 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:58.825865 18734 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.811070 15080 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:58.828474 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.828716 18734 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:58.832497 18734 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:58.832664 15054 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.832830 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 960) took 1372 ms. Trace:
I20260709 13:03:58.807633 18827 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:58.801151 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.833930 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.816483 15081 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:58.837785 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67871.3us
I20260709 13:03:58.830021 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.830195 18611 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.841626 15084 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:58.830287 18840 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:58.833423 15845 rpcz_store.cc:276] 0709 13:03:57.460327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.528105 (+ 67778us) service_pool.cc:224] Handling call
0709 13:03:58.661947 (+1133842us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:58.662170 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.829567 (+167397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.829597 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.829655 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:58.829710 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.832508 (+  2798us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:03:58.832558 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.832671 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389374592}
I20260709 13:03:58.830673 18835 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:58.833676 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.771733 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.839244 15842 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:03:58.849388 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1010) took 1304 ms. Trace:
I20260709 13:03:58.742255 15087 rpcz_store.cc:276] 0709 13:03:56.421939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.115170 (+1693231us) service_pool.cc:224] Handling call
0709 13:03:58.176420 (+ 61250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.397086 (+220666us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:58.544150 (+147064us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.732913 (+188763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.732990 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:58.733042 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.733110 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.733216 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5326,"prepare.queue_time_us":169487,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":12035,"raft.queue_time_us":208850,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":10699,"thread_start_us":411898,"threads_started":3}
I20260709 13:03:58.824682 18744 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:03:58.845055 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89008.8us
I20260709 13:03:58.848301 15080 raft_consensus.cc:2376] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:03:58.779062 15085 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:58.849897 15842 rpcz_store.cc:276] 0709 13:03:57.549228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.583590 (+ 34362us) service_pool.cc:224] Handling call
0709 13:03:58.713069 (+1129479us) spinlock_profiling.cc:243] Waited 917 us on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:03:58.849314 (+136245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109184}
I20260709 13:03:58.857048 15079 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:58.860204 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.865017 15086 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:58.866505 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.868643 15081 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:58.835930 18734 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:58.753846 18725 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.874670 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.876674 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:58.870486 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.881637 15843 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:58.882059 15840 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:03:58.877826 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:58.878558 15842 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:58.835592 18718 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:03:58.877447 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.879856 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:58.761564 18736 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:58.909655 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:58.911072 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56883.6us
I20260709 13:03:58.911279 15081 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:58.905699 18836 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:58.911517 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.853440 18832 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:03:58.853152 18793 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:58.913358 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2481) took 2145 ms. Trace:
I20260709 13:03:58.941408 15081 rpcz_store.cc:276] 0709 13:03:56.767769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.869630 (+2101861us) service_pool.cc:224] Handling call
0709 13:03:58.909755 (+ 40125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.912515 (+  2760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.912763 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15616}
I20260709 13:03:58.942744 15840 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:58.914405 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.956032 15079 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:03:58.958125 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 163874us
I20260709 13:03:58.960119 15845 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:58.962249 15080 raft_consensus.cc:2410] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:58.965929 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.971230 15087 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 5: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:03:58.973773 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65412.8us
I20260709 13:03:58.985684 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:58.915166 18838 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.975623 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2478) took 2190 ms. Trace:
I20260709 13:03:58.990226 15079 rpcz_store.cc:276] 0709 13:03:56.767037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.867575 (+2100538us) service_pool.cc:224] Handling call
0709 13:03:58.898471 (+ 30896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.956827 (+ 58356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.957100 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.993883 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:58.941124 18793 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.940650 18734 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:58.957890 18843 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.953018 18836 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:59.006563 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:03:58.994796 15081 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:03:59.000123 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83947.6us
I20260709 13:03:59.010571 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:59.011476 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133826us
I20260709 13:03:59.004891 15080 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:03:58.953433 18841 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:59.002436 18793 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.920279 18737 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:03:59.050189 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:59.085935 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:59.099380 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 144692us
I20260709 13:03:59.003660 18778 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.104101 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:03:59.111888 15202 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.720s (SENT)
I20260709 13:03:59.107348 18826 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:59.006454 18616 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:59.190127 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:59.119151 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 108655us
I20260709 13:03:59.113147 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:59.121511 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 413206us
I20260709 13:03:59.191041 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.125766 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:03:59.024302 18520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.028s	sys 0.050s
I20260709 13:03:59.017792 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:03:59.191977 18520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.029s	sys 0.050s
I20260709 13:03:59.181232 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:03:59.112988 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.182027 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:03:59.192000 15235 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:59.194216 15081 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:03:58.702438 15088 rpcz_store.cc:276] 0709 13:03:56.766573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.318178 (+1551605us) service_pool.cc:224] Handling call
0709 13:03:58.372372 (+ 54194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.411338 (+ 38966us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:58.494671 (+ 83333us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.628328 (+133657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.628386 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:58.628428 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.628492 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.628591 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22500,"prepare.queue_time_us":278075,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":99,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":82023,"thread_start_us":143257,"threads_started":2}
I20260709 13:03:59.186082 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:59.133610 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:03:59.132751 18378 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.196565 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.124048 18846 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:59.197816 15237 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:59.138202 18750 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:59.213096 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 76837.2us
I20260709 13:03:59.213857 18718 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:59.214208 18836 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.009085 18827 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.183286 18826 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.222034 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.227380 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.228441 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.201581 18378 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:59.251657 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.232352 15088 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:03:59.248335 15081 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:59.200618 18846 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.217402 18828 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.268801 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 903) took 2378 ms. Trace:
I20260709 13:03:59.287689 15079 rpcz_store.cc:276] 0709 13:03:56.890556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.138675 (+2248119us) service_pool.cc:224] Handling call
0709 13:03:59.234727 (+ 96052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.268304 (+ 33577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.268361 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.268428 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.268465 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.268543 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.268725 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.269583 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:59.278115 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.109884 18843 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.285296 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.298149 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.299374 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:59.301620 18855 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:03:59.301906 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:03:59.302294 18850 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:03:59.302758 15160 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:03:59.304009 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.304767 15088 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:59.268267 18801 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:59.305812 18801 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.306456 18801 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.306797 18771 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:59.307260 18771 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.307678 18771 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.284575 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.238024 18730 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:59.310242 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 889) took 2755 ms. Trace:
I20260709 13:03:59.200387 18778 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.237676 18826 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:59.349058 15848 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:03:59.305795 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:03:59.322192 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60517.6us
I20260709 13:03:59.352860 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.353893 15088 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:03:59.355037 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.355744 15088 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:59.253571 18378 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.317881 15082 rpcz_store.cc:276] 0709 13:03:56.554916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.115197 (+1560281us) service_pool.cc:224] Handling call
0709 13:03:58.150450 (+ 35253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:58.388645 (+238195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.388704 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:58.425165 (+ 36461us) log.cc:844] Serialized 52 byte log entry
0709 13:03:58.948996 (+523831us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:58.949053 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.949147 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.309944 (+360797us) raft_consensus.cc:1685] finished
0709 13:03:59.309972 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.310174 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1632,"raft.queue_time_us":208638,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":17370,"thread_start_us":594246,"threads_started":2,"wal-append.queue_time_us":558347}
I20260709 13:03:59.300485 18827 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.322716 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.328521 18730 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.361008 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:59.362469 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2479) took 2595 ms. Trace:
I20260709 13:03:59.324206 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.325088 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:03:59.369052 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 363469us
I20260709 13:03:59.289985 18846 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:59.300098 18744 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:59.345592 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:03:59.402387 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.309813 18775 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:59.296034 18725 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:59.392926 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.233417 18734 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.313148 18782 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.318979 18828 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.492000 18616 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:59.435806 14975 maintenance_manager.cc:419] P df53447e208f455db54704b341851d53: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033398
W20260709 13:03:59.521678 18866 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:03:59.539613 15844 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:03:59.492973 18868 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:59.438354 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 115745us
W20260709 13:03:59.489771 18862 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:03:59.514418 18861 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:59.489518 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 387576us
W20260709 13:03:59.538749 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58434.1us
I20260709 13:03:59.624657 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:03:59.625451 15202 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.843s (SENT)
I20260709 13:03:59.238487 18832 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.625921 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:59.626502 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.627115 15156 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:03:59.628131 18871 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:59.405926 15157 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:59.356711 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:03:59.630254 15088 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:59.489140 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 303066us
I20260709 13:03:59.631969 18736 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:03:59.633087 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.633971 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 751) took 1540 ms. Trace:
I20260709 13:03:59.550153 18778 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:59.634348 15231 rpcz_store.cc:276] 0709 13:03:58.093115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.114946 (+ 21831us) service_pool.cc:224] Handling call
0709 13:03:59.633734 (+1518788us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540005c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:03:59.633911 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477872896}
I20260709 13:03:59.568336 18843 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:59.637223 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:03:59.642278 18828 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:03:59.646085 18771 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.648398 18771 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:59.649142 18840 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:59.649569 18840 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:59.650593 18840 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.652043 18840 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.625461 15162 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:03:59.489245 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.539215 15083 rpcz_store.cc:276] 0709 13:03:56.767294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.863429 (+2096135us) service_pool.cc:224] Handling call
0709 13:03:59.115081 (+251652us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:03:59.115333 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.135295 (+ 19962us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:59.214787 (+ 79492us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.362078 (+147291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.362141 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:59.362191 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.362257 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.362411 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25292,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":48,"spinlock_wait_cycles":32816512,"thread_start_us":132104,"threads_started":1,"wal-append.queue_time_us":309941}
I20260709 13:03:59.545691 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 904) took 2640 ms. Trace:
I20260709 13:03:59.568078 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:03:59.508733 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 155418us
I20260709 13:03:59.656453 15085 rpcz_store.cc:276] 0709 13:03:56.905007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.544763 (+2639756us) service_pool.cc:224] Handling call
0709 13:03:59.545132 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.545359 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.545401 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.545455 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.545483 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.545530 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.545624 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.657204 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.396205 15161 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:59.407917 15086 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:03:59.582651 15845 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:59.408893 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.584645 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.655526 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.325799 18695 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:59.583884 18684 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.780s	user 0.008s	sys 0.004s
W20260709 13:03:59.584059 18800 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.802s	user 0.013s	sys 0.000s
I20260709 13:03:59.543210 18775 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.591248 18616 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:03:59.603243 18868 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:03:59.607805 18856 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.424s	user 0.005s	sys 0.000s
I20260709 13:03:59.608580 18782 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:03:59.615658 18877 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:59.715237 15083 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:59.716135 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1858) took 2073 ms. Trace:
I20260709 13:03:59.617408 18780 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:59.615703 18852 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.433s	user 0.002s	sys 0.010s
I20260709 13:03:59.621905 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.623968 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.625805 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.633996 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:59.577477 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 464373us
I20260709 13:03:59.587518 18730 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:59.490854 18860 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:03:59.631323 18378 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:03:59.662680 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.475157 18750 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:03:59.663687 15088 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
W20260709 13:03:59.665450 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 97453.7us
I20260709 13:03:59.559823 18793 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.673779 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.716526 15085 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:59.777465 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.809067 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1859) took 1767 ms. Trace:
I20260709 13:03:59.678807 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:59.809602 15085 rpcz_store.cc:276] 0709 13:03:58.042071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.666767 (+1624696us) service_pool.cc:224] Handling call
0709 13:03:59.716104 (+ 49337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.808774 (+ 92670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.809010 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.708630 15081 raft_consensus.cc:606] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:03:59.810910 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.791523 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:59.662822 18828 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:03:59.810789 15081 raft_consensus.cc:614] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:59.717769 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:59.812942 15162 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.810117 15084 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:03:59.809167 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:03:59.813717 15161 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:03:59.679100 15156 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:59.660905 18838 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:03:59.709227 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 899) took 2903 ms. Trace:
I20260709 13:03:59.716566 15083 rpcz_store.cc:276] 0709 13:03:57.645572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.656296 (+2010724us) service_pool.cc:224] Handling call
0709 13:03:59.667291 (+ 10995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.715814 (+ 48523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.716065 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.715595 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:03:59.821627 15080 rpcz_store.cc:276] 0709 13:03:56.805438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.015674 (+2210236us) service_pool.cc:224] Handling call
0709 13:03:59.193263 (+177589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.218758 (+ 25495us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:59.357512 (+138754us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.708085 (+350573us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.708379 (+   294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.708437 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:59.708499 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.708988 (+   489us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.709156 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16006,"prepare.queue_time_us":106458,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":16079,"raft.queue_time_us":15535,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":106770,"thread_start_us":150558,"threads_started":1}
I20260709 13:03:59.826182 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.665926 18736 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.671664 18775 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.850647 15163 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:59.714570 18611 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:59.791036 18867 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:59.831907 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:59.830526 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:03:59.840181 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 905) took 1485 ms. Trace:
I20260709 13:03:59.878994 15088 rpcz_store.cc:276] 0709 13:03:58.347896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.813172 (+1465276us) service_pool.cc:224] Handling call
0709 13:03:59.814366 (+  1194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.819351 (+  4985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.819403 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.832504 (+ 13101us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:59.832597 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.832691 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.833101 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877888}
I20260709 13:03:59.714949 18877 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:03:59.716630 18868 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:59.829967 18856 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:03:59.875258 15202 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.018s (SENT)
I20260709 13:03:59.880564 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:03:59.881533 15159 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:59.879040 15160 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.809930 18801 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.668818 18826 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.824429 18684 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:59.887553 15080 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 8. Ops: []
I20260709 13:03:59.902673 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1862) took 1306 ms. Trace:
I20260709 13:03:59.903138 15080 rpcz_store.cc:276] 0709 13:03:58.595867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.854220 (+1258353us) service_pool.cc:224] Handling call
0709 13:03:59.887329 (+ 33109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.902327 (+ 14998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.902598 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.671640 18800 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.891253 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:03:59.891341 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 186733us
I20260709 13:03:59.883400 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 906) took 1510 ms. Trace:
I20260709 13:03:59.909619 15839 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:59.909938 15084 rpcz_store.cc:276] 0709 13:03:58.373569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.814805 (+1441236us) service_pool.cc:224] Handling call
0709 13:03:59.846659 (+ 31854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.876943 (+ 30284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.876991 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.883081 (+  6090us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:59.883132 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.883215 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.883341 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.893929 15161 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:59.886862 18877 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.877336 18611 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:59.896140 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 262283us
W20260709 13:03:59.897182 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 218468us
I20260709 13:03:59.891150 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1865) took 1027 ms. Trace:
W20260709 13:03:59.914410 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70631.1us
I20260709 13:03:59.922705 15085 rpcz_store.cc:276] 0709 13:03:58.864317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.875831 (+1011514us) service_pool.cc:224] Handling call
0709 13:03:59.890674 (+ 14843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.890827 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.890858 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.890895 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:59.890925 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.890973 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.891100 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.885977 18867 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:03:59.824576 18828 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.892932 18782 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:59.923846 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:59.927284 15082 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:03:59.932912 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 266100us
W20260709 13:03:59.933780 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 100056us
I20260709 13:03:59.936872 15080 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:59.845106 18838 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:03:59.921474 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:03:59.895608 18772 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.944350 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1867) took 1039 ms. Trace:
I20260709 13:03:59.948740 15080 rpcz_store.cc:276] 0709 13:03:58.905323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.904112 (+998789us) service_pool.cc:224] Handling call
0709 13:03:59.920026 (+ 15914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.944091 (+ 24065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.944286 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.939661 15845 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:03:59.946358 15084 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:03:59.951748 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:03:59.959642 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:03:59.922013 18836 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.926640 18611 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:00.018772 15202 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.151s (SENT)
W20260709 13:04:00.050887 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 91364.5us
I20260709 13:03:59.885586 18736 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:03:59.941542 18867 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.045130 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1000) took 1245 ms. Trace:
W20260709 13:04:00.047319 18827 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 13:04:00.045318 15080 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:04:00.049145 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.055236 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:00.049365 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:03:59.951702 18772 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:00.018855 18725 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.052224 15844 rpcz_store.cc:276] 0709 13:03:58.799699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.800932 (+  1233us) service_pool.cc:224] Handling call
0709 13:04:00.031305 (+1230373us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54006d6fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:00.045061 (+ 13756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1615827200}
I20260709 13:03:59.945370 18792 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:04:00.049463 18836 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.899833 18616 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.049878 18891 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:04:00.086465 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 271313us
I20260709 13:04:00.057682 18844 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.054339 18841 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:00.057859 18772 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.060491 18725 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:00.090597 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 61080.8us
I20260709 13:04:00.092371 15160 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.064361 18612 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:00.063967 18822 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:00.083573 18836 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.099385 15084 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:04:00.114645 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 193251us
I20260709 13:04:00.089977 18844 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:00.098068 18780 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.099133 18725 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.121443 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.059091 18750 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:00.133278 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 189322us
W20260709 13:04:00.126396 18859 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.883s	user 0.003s	sys 0.007s
I20260709 13:04:00.123659 18844 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:00.175571 15202 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.166s (SENT)
I20260709 13:04:00.128599 18780 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:00.178272 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44116.2us
I20260709 13:04:00.130199 18378 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:00.178910 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84660.8us
I20260709 13:04:00.125321 18782 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.115200 18889 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:04:00.195159 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1866) took 1310 ms. Trace:
I20260709 13:04:00.182950 18780 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.199483 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 908) took 1813 ms. Trace:
I20260709 13:04:00.118289 18891 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:04:00.185240 18801 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:00.192593 15202 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.509s (SENT)
W20260709 13:04:00.197919 18881 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:04:00.225162 15088 rpcz_store.cc:276] 0709 13:03:58.885178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.880065 (+994887us) service_pool.cc:224] Handling call
0709 13:03:59.895883 (+ 15818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.916102 (+ 20219us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:59.951928 (+ 35826us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.194546 (+242618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.194722 (+   176us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:00.194796 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.194898 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.195075 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32608,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3845,"raft.queue_time_us":17982,"thread_start_us":121265,"threads_started":1}
W20260709 13:04:00.207717 18884 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:04:00.233708 15083 rpcz_store.cc:276] 0709 13:03:58.385848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.823648 (+1437800us) service_pool.cc:224] Handling call
0709 13:03:59.838982 (+ 15334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.884655 (+ 45673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.884703 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.888379 (+  3676us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.092575 (+204196us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.101496 (+  8921us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.103067 (+  1571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.103154 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.199236 (+ 96082us) raft_consensus.cc:1685] finished
0709 13:04:00.199255 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.199421 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4559,"prepare.queue_time_us":9323,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":48459,"spinlock_wait_cycles":34377984,"thread_start_us":172493,"threads_started":1,"wal-append.queue_time_us":249699}
I20260709 13:04:00.191553 18826 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:00.346113 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 172287us
I20260709 13:04:00.347967 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2057) took 1103 ms. Trace:
I20260709 13:04:00.349732 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 925) took 1161 ms. Trace:
I20260709 13:04:00.350293 15082 rpcz_store.cc:276] 0709 13:03:59.188415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.363131 (+174716us) service_pool.cc:224] Handling call
0709 13:04:00.349505 (+986374us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540014df54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:00.349660 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595517312}
I20260709 13:04:00.348491 15163 rpcz_store.cc:276] 0709 13:03:59.244352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.244526 (+   174us) service_pool.cc:224] Handling call
0709 13:03:59.286246 (+ 41720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.880085 (+593839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.347730 (+467645us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:00.347911 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275406208}
W20260709 13:04:00.375131 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 257565us
I20260709 13:04:00.255505 18900 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:00.380107 18900 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:00.380645 18900 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.382278 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.389164 18836 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:00.349834 18800 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.390851 15158 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:00.376418 18838 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:00.401481 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 220691us
W20260709 13:04:00.402051 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.403987 18793 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:00.404213 15202 leader_election.cc:336] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.860s (SENT)
I20260709 13:04:00.404727 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.353824 18828 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.404803 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.418351 18822 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:00.405756 18800 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.406527 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:00.426270 15162 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.005s
I20260709 13:04:00.426704 15162 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:00.420917 15202 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.045s (SENT)
W20260709 13:04:00.354488 18887 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:04:00.430642 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:00.432578 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.391665 18836 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.329921 18736 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:00.417083 18883 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.362s	user 0.005s	sys 0.008s
W20260709 13:04:00.433107 18885 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.247s	user 0.000s	sys 0.008s
W20260709 13:04:00.430897 18881 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:04:00.427799 18865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.007s	sys 0.026s
W20260709 13:04:00.432152 18730 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.004s
I20260709 13:04:00.435815 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.439226 18822 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:00.451609 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.414172 18877 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.407704 18772 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:00.447479 18865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.007s	sys 0.027s
I20260709 13:04:00.410921 18611 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.413216 18725 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:00.411795 18867 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.451097 18889 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.471926 15087 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:00.473341 15842 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:00.475710 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2495) took 1605 ms. Trace:
I20260709 13:04:00.476082 15086 rpcz_store.cc:276] 0709 13:03:58.870684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.883168 (+ 12484us) service_pool.cc:224] Handling call
0709 13:03:59.103909 (+220741us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:04:00.475522 (+1371613us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540014d554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:00.475654 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547989120}
I20260709 13:04:00.449704 18730 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.477092 18897 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:00.446038 18782 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.469321 18891 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:00.506204 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 70459.9us
I20260709 13:04:00.508949 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.502592 18611 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:00.505668 18868 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:04:00.506170 18908 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.506703 18822 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:00.549218 18908 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.552372 18908 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:00.480244 18750 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.553198 18750 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:00.556128 18750 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.561084 18868 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.562357 15846 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:00.470307 18889 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:00.516592 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1857) took 2958 ms. Trace:
I20260709 13:04:00.566525 15079 rpcz_store.cc:276] 0709 13:03:57.595188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.639141 (+2043953us) service_pool.cc:224] Handling call
0709 13:03:59.639485 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:59.639844 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.639870 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:59.640458 (+   588us) log.cc:844] Serialized 52 byte log entry
0709 13:03:59.877814 (+237356us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.878128 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.880500 (+  2372us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.516251 (+635751us) raft_consensus.cc:1685] finished
0709 13:04:00.516294 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.516511 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15164,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":108,"thread_start_us":236844,"threads_started":1,"wal-append.queue_time_us":441839}
W20260709 13:04:00.517132 18840 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.865s	user 0.007s	sys 0.008s
W20260709 13:04:00.525563 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47606.8us
I20260709 13:04:00.526683 18730 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.502622 18772 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:00.542505 18877 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:00.575352 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.540216 18780 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:00.576359 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 67556.1us
I20260709 13:04:00.578524 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.573967 18889 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:00.577370 18846 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.001s
I20260709 13:04:00.585515 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:00.579058 18897 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.579005 18891 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.593849 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:00.587378 18775 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.000s
I20260709 13:04:00.597258 18725 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.597362 15086 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:04:00.601936 15086 pending_rounds.cc:85] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:04:00.602002 18898 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.602275 15086 pending_rounds.cc:107] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:04:00.602545 18898 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:00.602756 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.603027 18898 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.603395 15081 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:00.604449 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.602566 15086 raft_consensus.cc:2889] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:00.599866 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.587086 18868 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:00.592697 18908 raft_consensus.cc:2764] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:00.608017 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.592567 18611 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.611140 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.609357 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.609568 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.610435 15083 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:00.612273 15088 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:00.612185 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.613149 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.614744 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.617234 18867 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.618175 18891 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.620472 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 901) took 3776 ms. Trace:
I20260709 13:04:00.620433 18844 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:00.618188 18908 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.621485 18908 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:00.621953 18908 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.623268 15086 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:00.624513 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.625249 15086 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:00.626012 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.621085 15087 rpcz_store.cc:276] 0709 13:03:56.843742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.098899 (+2255157us) service_pool.cc:224] Handling call
0709 13:03:59.181397 (+ 82498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.472382 (+1290985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.620225 (+147843us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400305f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:00.620410 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2967500160}
I20260709 13:04:00.626549 15086 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:00.627187 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.627815 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.628458 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.629061 15845 raft_consensus.cc:2376] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8. Re-sending same reply.
I20260709 13:04:00.627948 15080 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:00.632551 15085 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:00.633167 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 286321us
I20260709 13:04:00.637267 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.639068 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.639648 15157 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.645031 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.611583 18611 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.646030 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.647795 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.651211 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.651870 15228 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:00.654182 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:00.655117 15202 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.516s (SENT)
I20260709 13:04:00.655236 15228 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:00.656459 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.657109 15237 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.657464 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.657768 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.658166 15229 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.658411 15236 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:00.660362 18736 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:00.660563 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.650045 18725 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:00.589888 18846 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:00.662230 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 76813us
I20260709 13:04:00.664378 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.664690 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.664569 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.665277 15845 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.664952 15156 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:00.666330 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.667694 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.668196 15230 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:04:00.669615 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.670138 15162 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.670257 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:00.673722 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.674322 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.677222 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.675904 15231 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:00.679349 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.678580 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.677253 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.678409 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.684703 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.686152 18867 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.686733 15159 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:00.686892 15841 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:00.687470 15841 raft_consensus.cc:606] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.688472 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.605856 18775 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.686481 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:00.662890 18611 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.701493 18780 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.730141 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:00.722084 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:00.701309 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 319151us
I20260709 13:04:00.671854 18792 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.708155 15157 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:00.709343 15161 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.721935 15237 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:00.669250 18725 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:00.729537 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.749147 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.731550 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:00.732005 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.733517 15845 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:00.683043 18897 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:00.675416 18913 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:00.663681 18877 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:04:00.686585 18867 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:00.756800 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79583.4us
I20260709 13:04:00.757396 15840 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:00.756618 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.760255 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.785557 15162 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:04:00.787531 15844 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:00.761642 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.762982 15847 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:00.788750 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.763942 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.730327 18378 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:00.769017 18725 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:00.764637 18908 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.772552 18800 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:00.803944 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:00.804625 15081 raft_consensus.cc:606] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.804780 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.779254 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.781181 15083 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:04:00.806452 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 129852us
W20260709 13:04:00.760860 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 81585.7us
I20260709 13:04:00.758514 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.789095 15157 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.807750 15841 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 5.
I20260709 13:04:00.762317 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.808964 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.809916 15845 raft_consensus.cc:2410] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:00.810443 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.788143 15236 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:00.793027 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 130064us
I20260709 13:04:00.790813 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.799472 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:00.740489 18792 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:00.812634 15128 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:00.812278 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:00.762506 18913 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:00.816138 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.818782 15155 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:00.784888 18695 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:00.818068 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.787822 18838 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:00.815093 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:00.820989 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.815462 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:00.814981 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.821873 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.790892 18867 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.834175 18846 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:00.806216 18897 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.836946 18868 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:00.836866 18897 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.818982 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.819109 15236 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.811290 15845 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:00.740165 18900 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.824534 18844 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.830125 18908 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:00.831453 18909 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:00.833280 18775 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:00.842391 15157 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:00.820930 18913 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:00.849066 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:00.850554 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.860535 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.851557 15162 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:00.863796 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44894.1us
I20260709 13:04:00.864167 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:00.864492 18880 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.811s	user 0.000s	sys 0.008s
I20260709 13:04:00.865067 18868 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:00.865453 18868 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:00.865826 18868 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.866410 18612 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:00.866745 15236 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.853286 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:00.867475 15230 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.867583 18730 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:00.868181 18730 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.850320 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:00.814596 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 131741us
W20260709 13:04:00.855729 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.819591 18792 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.866818 18612 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.854626 18877 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:00.857491 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63053.9us
I20260709 13:04:00.874261 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.869886 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.850136 15232 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:00.880539 15159 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:00.881191 15228 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:00.882409 15236 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:00.883594 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.883288 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.889607 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 976) took 1246 ms. Trace:
I20260709 13:04:00.889700 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.890552 15232 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.890306 15842 rpcz_store.cc:276] 0709 13:03:59.643213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.663525 (+ 20312us) service_pool.cc:224] Handling call
0709 13:04:00.889423 (+1225898us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54004b55d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:00.889539 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528770304}
I20260709 13:04:00.893285 15230 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:00.893483 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:00.872831 18378 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:00.828614 18889 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:00.895553 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 82977.7us
I20260709 13:04:00.894927 15162 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:00.874433 18844 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:00.898936 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.905454 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:00.900024 15052 connection.cc:256] RPC call timeout handler was delayed by -2.3026027099999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 13:04:00.908048 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:00.814106 18922 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:04:00.909322 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:00.867583 18893 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:00.882362 18838 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:00.885249 18841 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.884439 18909 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.898833 18721 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:00.918076 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.896373 18876 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.923736 18772 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:00.924345 18923 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:00.925601 18922 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:00.898176 18828 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.900671 18793 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:00.929890 18844 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:00.913022 18782 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:00.930722 18826 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.930524 18915 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:04:00.931205 15052 connection.cc:256] RPC call timeout handler was delayed by -1.8760027099999661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
I20260709 13:04:00.930936 18838 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.901513 18906 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.932621 15228 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.901022 18771 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.936960 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56749.3us
I20260709 13:04:00.898720 18736 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:04:00.929736 18921 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:04:00.913365 18891 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:00.909497 18378 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:00.945464 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:00.945521 15203 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.516s (SENT)
I20260709 13:04:00.917375 18775 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:00.957333 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:00.949331 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48108.4us
W20260709 13:04:00.950189 15052 connection.cc:256] RPC call timeout handler was delayed by -1.8217431140000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:04:00.918855 18893 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.938416 18876 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.952358 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:01.009802 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.011164 15083 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:00.952035 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51511.4us
I20260709 13:04:00.942112 18843 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:00.954242 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.955232 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:00.940819 18772 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:00.941063 18828 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:00.940578 18793 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:00.907747 18730 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.934027 18841 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.927440 18832 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:04:00.912786 18892 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:00.927475 18684 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:00.946362 18721 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:00.971386 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.970896 15842 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:00.986630 15079 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:00.987869 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:00.973623 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.965123 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:01.055352 15087 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:00.968106 15228 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.986409 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2502) took 1031 ms. Trace:
I20260709 13:04:01.056468 15085 rpcz_store.cc:276] 0709 13:03:59.954689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.047392 (+ 92703us) service_pool.cc:224] Handling call
0709 13:04:00.986212 (+938820us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:00.986349 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322281728}
I20260709 13:04:00.967960 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.995644 15232 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:00.996392 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:01.059998 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63694.5us
I20260709 13:04:00.948055 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:00.998446 15052 connection.cc:256] RPC call timeout handler was delayed by -1.3150431140001047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23090000000000002s for a response
W20260709 13:04:01.005568 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 86263.5us
I20260709 13:04:00.987990 15839 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
I20260709 13:04:00.997890 15840 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 5.
W20260709 13:04:00.962806 18919 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:04:00.971701 18929 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:01.030383 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.030488 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:00.955778 18782 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:00.950448 18826 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.033955 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:00.968215 18771 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.066308 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:00.957620 18736 consensus_queue.cc:1243] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:01.013881 18876 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.062949 15082 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:01.092439 15841 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:00.945861 18898 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.081496 18801 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.094659 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.037745 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:01.026212 18828 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.103542 15233 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:01.027724 18921 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:01.031522 18793 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.974735 18913 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.106076 15083 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.008008 18827 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:04:01.113500 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:00.997797 18611 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.095191 15237 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.037021 18889 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.082810 18877 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:04:01.113514 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47345.3us
I20260709 13:04:01.010732 18775 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.098325 18892 consensus_queue.cc:1243] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:01.139632 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 75504.4us
I20260709 13:04:00.957391 18906 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.093927 18929 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:01.028824 18792 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:00.963133 18909 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.107378 18900 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.160113 15848 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.186228 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:01.210729 15080 raft_consensus.cc:606] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:01.211074 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 96646.2us
I20260709 13:04:01.211364 15080 raft_consensus.cc:614] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:01.142328 18921 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:01.154772 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60198.2us
I20260709 13:04:01.145378 18832 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:01.155949 18772 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.175467 18378 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.193730 18906 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.104625 18844 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.138397 18801 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.137758 18928 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:04:01.194995 18929 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.242722 18929 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:01.196918 18898 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.199528 15158 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:01.200141 18867 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.190191 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:01.262192 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.175187 18843 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:01.218006 15083 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:01.219615 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.221460 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.228029 15237 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:01.186519 18891 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:01.228165 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:01.236528 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 291014us
I20260709 13:04:01.233171 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.241384 18771 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:01.258797 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:01.203387 15845 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:01.288269 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:01.286525 15203 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.585s (SENT)
I20260709 13:04:01.289544 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:01.286511 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 279208us
W20260709 13:04:01.289567 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 99433.9us
W20260709 13:04:01.290079 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68719.4us
I20260709 13:04:01.296770 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:01.246556 18772 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.299398 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:01.302587 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43968.6us
I20260709 13:04:01.201643 18721 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.308190 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.283058 18843 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:01.312376 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 71339.9us
I20260709 13:04:01.287179 18877 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:01.291913 18725 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.318015 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:01.312799 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 283551us
I20260709 13:04:01.298298 18892 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:01.289237 18736 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.220762 18943 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:04:01.306037 18684 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:01.231022 18782 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.335129 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.337704 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:01.317083 18843 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.339016 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.296597 18906 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:04:01.340343 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 53869.7us
I20260709 13:04:01.303516 18832 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:01.322755 18892 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:01.356196 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48122.4us
I20260709 13:04:01.347556 18775 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.348191 18898 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:01.358150 18916 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:01.358399 18898 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:01.331336 18827 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:01.346047 15203 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.643s (SENT)
I20260709 13:04:01.359862 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.333206 18792 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:01.342592 18877 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:01.357720 18939 raft_consensus.cc:2764] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:04:01.358739 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4010b839395412ca61c77e701fcb965"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:01.418089 15080 raft_consensus.cc:606] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.418480 15080 raft_consensus.cc:614] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:01.343942 18684 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:01.345649 18841 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:01.422785 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.424084 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 101422us
I20260709 13:04:01.430042 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.358645 18916 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.353461 18826 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.436405 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:01.437556 15085 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:01.437952 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100306us
I20260709 13:04:01.328444 18725 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.362361 18892 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:01.458899 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.303923 18921 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:01.453533 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 98542.8us
I20260709 13:04:01.470669 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:01.471136 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 971) took 2209 ms. Trace:
I20260709 13:04:01.471511 15231 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:04:01.474085 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44167.7us
I20260709 13:04:01.471516 15846 rpcz_store.cc:276] 0709 13:03:59.262079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.262308 (+   229us) service_pool.cc:224] Handling call
0709 13:03:59.301428 (+ 39120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.562865 (+1261437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.470881 (+908016us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400580054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:01.471078 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2899212032}
I20260709 13:04:01.427513 18611 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.436954 18916 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.478605 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:01.448057 18913 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:01.451355 18792 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:01.417665 18736 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.488600 18944 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:01.489121 18931 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:04:01.489488 18941 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:01.359457 18898 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.463619 18827 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:01.356101 18891 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.472194 18943 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:01.512055 18943 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.514724 18943 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.472404 18939 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:01.516618 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:01.517174 18939 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.517580 15086 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:01.495909 18895 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.251s	user 0.000s	sys 0.015s
I20260709 13:04:01.483211 18930 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:01.521871 15087 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:01.506254 18844 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:01.525565 18948 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:01.526727 18750 raft_consensus.cc:2764] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:04:01.507141 18868 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.641s	user 0.007s	sys 0.006s
I20260709 13:04:01.538146 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:34666
W20260709 13:04:01.550081 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 230475us
I20260709 13:04:01.538934 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:01.542980 15081 raft_consensus.cc:606] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.556198 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.556738 15081 raft_consensus.cc:614] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:01.510234 18944 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:01.557134 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:01.519683 18946 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:01.562238 15846 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:01.568796 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:01.577625 15843 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:01.582744 18922 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:01.592554 18822 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:01.593127 18822 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:01.594136 18822 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.594980 18725 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.595552 18822 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:01.595860 18940 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:01.553855 18868 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.598834 18939 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:01.599924 18843 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.579802 18893 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.603067 18843 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:01.603309 18893 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.572155 18938 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:01.584623 18922 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:01.572134 18949 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:01.537928 18954 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.610046 18954 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.565443 18838 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:01.613238 18838 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.613281 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:01.613763 18838 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.614343 15085 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:04:01.615490 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1871) took 1021 ms. Trace:
I20260709 13:04:01.615939 15079 rpcz_store.cc:276] 0709 13:04:00.594466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.601022 (+  6556us) service_pool.cc:224] Handling call
0709 13:04:00.986376 (+385354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.053936 (+ 67560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.615288 (+561352us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b5400306950. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:01.615428 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869196032}
I20260709 13:04:01.521895 18684 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.618208 15842 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.620831 18952 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:01.621694 15848 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:01.622045 18611 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:01.538265 18867 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:01.622501 18611 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:01.623237 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:01.623385 18611 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.624831 18611 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.625524 18695 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:01.626072 18695 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:01.565685 18921 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:01.627470 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.627079 18695 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.630505 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:01.631883 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:01.632705 15843 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:01.633435 18944 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.533063 18801 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.634065 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:01.643210 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:01.624485 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:01.612550 18954 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.636320 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:01.643441 15158 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.645605 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.647960 15155 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.649650 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:01.658774 15232 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:01.635504 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:01.661214 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:01.665602 15162 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.666206 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:01.666821 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:01.695114 15848 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 LEARNER]: Advancing to term 8
W20260709 13:04:01.707149 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46829.5us
I20260709 13:04:01.710590 15160 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:04:01.712935 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 67389.9us
I20260709 13:04:01.714277 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.716274 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:01.534576 18378 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
W20260709 13:04:01.721304 18955 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:04:01.734755 15161 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:01.722432 18957 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:04:01.727293 15232 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:01.715770 18611 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:01.659199 18771 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.659162 18895 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:04:01.736799 15230 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:01.737442 15843 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:04:01.737648 15842 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:01.705236 18832 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:01.793792 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:01.739061 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.598836 18868 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:01.751525 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:01.804807 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53374.6us
W20260709 13:04:01.710505 18793 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:04:01.787386 15159 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:01.799708 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 499040us
I20260709 13:04:01.729627 18772 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:01.788686 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:01.810616 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 464570us
W20260709 13:04:01.812513 15203 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.730s (SENT)
I20260709 13:04:01.813964 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:01.832445 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 93476.6us
W20260709 13:04:01.797658 18828 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 13:04:01.728966 18827 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.790086 18946 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:01.857095 18877 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:01.788394 18611 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:01.730911 18954 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:04:01.832351 18909 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:01.791713 18867 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:04:01.790380 18616 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:04:01.896500 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 340349us
I20260709 13:04:01.869915 15848 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEARNER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:01.796326 18832 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.787925 18801 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:04:01.795786 18792 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:04:01.804171 18952 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:04:01.800963 18771 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:01.920558 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126851us
I20260709 13:04:01.831504 18793 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.930855 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:01.937335 18959 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:01.939323 18771 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:01.940317 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 127849us
W20260709 13:04:01.940990 15203 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.841s (SENT)
I20260709 13:04:01.941401 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:01.879678 18889 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:01.934151 18949 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:04:01.949301 18976 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:01.949751 18976 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:01.950474 18736 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:01.951058 18736 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:01.951874 18736 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:01.952209 18736 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:01.924223 18832 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.955016 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:01.955935 15843 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:01.955973 15844 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:01.872821 18828 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:01.957605 15847 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:04:01.958240 18942 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.167s
I20260709 13:04:01.917997 18611 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.950757 18976 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:01.969040 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:01.925359 18378 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:02.004370 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 152497us
I20260709 13:04:01.949805 18929 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:02.012054 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:02.013074 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.014204 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:01.959554 18909 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:02.023849 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82891.6us
I20260709 13:04:02.010293 18889 consensus_queue.cc:1243] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:01.982594 18822 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:04:02.014104 18929 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.935551 18892 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.028331 18923 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:02.015019 18378 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:02.012900 18963 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:04:02.050676 15203 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.667s (SENT)
I20260709 13:04:02.058198 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.058703 15846 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:02.023324 18949 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:02.051113 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:01.960143 18966 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:04:02.056588 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1020) took 1215 ms. Trace:
I20260709 13:04:02.162073 15845 rpcz_store.cc:276] 0709 13:04:00.841426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.841574 (+   148us) service_pool.cc:224] Handling call
0709 13:04:01.203078 (+361504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.289116 (+ 86038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.056380 (+767264us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54006f0250. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:02.056530 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799341824}
I20260709 13:04:02.165088 15844 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:02.049659 18929 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:02.052281 18921 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:02.012352 18736 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:02.170929 18967 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:04:02.173252 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122272us
I20260709 13:04:02.173307 18942 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:02.055757 18923 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:02.174012 18971 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:04:02.032451 18877 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.174484 18822 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:02.181888 18892 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.183219 18909 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.184121 18898 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:02.217077 18898 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:02.217521 18793 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.217622 18942 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:02.218616 18916 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.000s
I20260709 13:04:02.218283 18898 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.219036 18916 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.220055 18942 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.184440 18378 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.185415 18965 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.221450 18965 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:02.186461 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 135831us
W20260709 13:04:02.193693 18962 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:02.193490 18832 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.023561 18959 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 13:04:02.185856 18963 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:02.057592 18828 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:02.187013 18876 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.043s	user 0.013s	sys 0.000s
I20260709 13:04:02.210647 18923 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:02.225279 15203 leader_election.cc:336] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.787s (SENT)
I20260709 13:04:02.242774 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.225159 18736 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:02.220082 18898 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.235378 18832 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:02.209777 18838 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.004s
I20260709 13:04:02.224790 18965 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:02.265900 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 934) took 1270 ms (client timeout 1612 ms). Trace:
W20260709 13:04:02.266366 15083 rpcz_store.cc:269] 0709 13:04:00.995557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.011726 (+ 16169us) service_pool.cc:224] Handling call
0709 13:04:02.265727 (+1254001us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b5400306e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:02.265832 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6521600}
I20260709 13:04:02.250571 18922 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:04:02.271912 18846 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:02.247857 18876 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.272346 18846 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:02.273952 18876 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.273530 18846 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.270843 15846 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:02.271385 15847 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:02.271327 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:02.279245 18911 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:02.279642 18911 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:02.261329 18923 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.276067 18838 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.297045 18986 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:02.297312 18957 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:02.306656 18838 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.280447 18911 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:02.307567 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.312582 18946 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.630663 18826 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:02.630949 18946 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.631297 18826 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.632323 18841 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:02.632550 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:02.276558 18611 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.632905 18841 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.633704 18826 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.634858 18611 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.634858 18841 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.635551 15844 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.635684 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:02.636449 15083 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:02.488478 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.637264 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.589206 18929 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.354014 18921 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.638576 18921 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.639467 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:02.642102 15161 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:02.643935 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:02.644860 15846 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:02.645908 18911 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:02.620389 18897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.628s	user 0.027s	sys 0.041s
W20260709 13:04:02.646775 18897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.693s	user 0.027s	sys 0.042s
I20260709 13:04:02.644129 15083 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:02.649108 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:02.652606 18946 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.311230 18772 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.343s
I20260709 13:04:02.654120 18923 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.658445 15846 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
W20260709 13:04:02.310540 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 474608us
I20260709 13:04:02.662303 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.663189 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:02.664535 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:02.666029 15086 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [5.4-7.5]
W20260709 13:04:02.629643 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 404369us
I20260709 13:04:02.343019 18916 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.670516 15083 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:02.668571 15203 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.950s (SENT)
I20260709 13:04:02.670892 18822 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.670693 18897 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.627557 18951 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.006s	sys 0.018s
I20260709 13:04:02.672763 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.679612 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:02.628822 18976 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:02.663676 18838 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.664073 18929 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.686079 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:02.629693 18846 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.306710 18725 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:02.683954 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:02.693585 15082 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:02.678377 18793 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:02.671731 18975 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:02.680476 18951 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.006s	sys 0.025s
I20260709 13:04:02.690994 15083 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:02.692260 15203 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.800s (SENT)
I20260709 13:04:02.672391 18827 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:02.695380 15080 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:02.701727 15082 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:02.673640 18946 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:02.672077 18695 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:04:02.680773 18900 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:04:02.671446 18832 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:04:02.678836 18915 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:04:02.674187 18822 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.719192 18822 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:02.709949 15087 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.707854 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.704968 18975 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:02.686861 18897 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.709717 18946 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.710599 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:02.799748 18695 consensus_queue.cc:1243] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:02.702168 18963 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:02.690297 18838 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:02.802331 15080 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:02.710829 18793 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:02.809309 18946 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:02.814643 18822 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:02.813879 18915 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.836858 18987 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:04:02.839560 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 147318us
W20260709 13:04:02.691530 15052 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff3342a5b2ad49e2b252d7cd47b9851d. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:02.847438 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 177898us
I20260709 13:04:02.848410 15052 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:02.853240 18827 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:02.855268 18988 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:04:02.857180 18793 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:02.861568 18989 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 13:04:02.863534 18990 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:04:02.863641 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.505s (SENT) [suppressed 40 similar messages]
I20260709 13:04:02.835237 18915 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.874473 15087 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:02.850451 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:02.884383 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:02.890866 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:02.890918 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:02.892549 15847 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:02.895358 15082 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:04:02.896282 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:02.894332 15083 pending_rounds.cc:85] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:04:02.897908 15083 pending_rounds.cc:107] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:02.898236 15080 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:04:02.898249 15083 raft_consensus.cc:2889] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.885288 18827 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:02.901721 15203 connection.cc:256] RPC call timeout handler was delayed by -1.5527264620000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
I20260709 13:04:02.888628 18772 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.905422 15203 connection.cc:256] RPC call timeout handler was delayed by -1.5369393799999216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260709 13:04:02.906042 15203 connection.cc:256] RPC call timeout handler was delayed by -1.5315145139999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 13:04:02.887650 18841 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:02.906419 15203 connection.cc:256] RPC call timeout handler was delayed by -1.4330264620000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:04:02.906842 15085 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
W20260709 13:04:02.906895 15203 connection.cc:256] RPC call timeout handler was delayed by -1.3380394900000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:04:02.910934 18992 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:04:02.912025 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:02.913309 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 111838us
W20260709 13:04:02.902336 18985 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:04:02.915693 15813 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.857s (SENT)
I20260709 13:04:02.921376 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:02.888177 18877 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:04:02.904978 18827 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.875324 18975 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:02.953204 18827 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:02.996438 15842 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:02.999611 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 136007us
I20260709 13:04:02.914354 18772 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:02.999545 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83873.1us
I20260709 13:04:02.981621 18826 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:02.983801 18937 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:02.920965 18841 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:03.015079 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 107521us
I20260709 13:04:02.990152 18897 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:02.998042 18838 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.020637 15237 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:03.020129 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.023844 15230 raft_consensus.cc:2368] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:04:03.026029 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.007927 18775 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.032356 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.033313 15228 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 6: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:03.037273 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.037786 15229 raft_consensus.cc:1217] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-7.7]   Dedup: 4.6->[7.7-7.7]
I20260709 13:04:03.038087 15233 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:03.038579 15229 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [4.6-7.7]
I20260709 13:04:03.038940 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.041677 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.042527 15229 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:04:03.043319 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.039733 15234 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:03.016562 18822 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:03.047035 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.020987 18929 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.018750 18963 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.049259 15237 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 5.
I20260709 13:04:03.014583 18951 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:04:03.052742 18841 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.053150 18838 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:03.053431 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:03.053900 18822 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.116132 18822 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.118042 18772 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.051973 18915 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.015810 18826 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.081898 15161 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:03.088733 18736 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:03.090128 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.131249 15237 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:03.091427 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.092061 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.133950 15232 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.072996 19001 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:03.102427 15203 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.941s (SENT)
I20260709 13:04:03.138250 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:03.127720 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43989.3us
I20260709 13:04:03.133307 15234 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
W20260709 13:04:03.144946 15203 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.074s (SENT)
I20260709 13:04:03.145540 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.137281 19001 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.147125 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.149253 15161 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:03.037303 18775 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.149771 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.150354 15160 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.063637 18929 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:03.154135 18994 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:04:03.154778 18775 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.157401 18736 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:03.162717 15161 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:04:03.161092 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:03.171669 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.077420 18963 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.194895 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:03.195569 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50673.3us
I20260709 13:04:03.195988 15161 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:03.196365 15203 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.158s (SENT)
I20260709 13:04:03.196826 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.197737 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.148068 18822 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:03.182519 18750 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:03.199828 18923 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:03.203550 18963 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:03.204349 18976 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.000s
I20260709 13:04:03.128963 18838 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.216063 18838 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:03.204937 15203 leader_election.cc:336] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.226s (SENT)
I20260709 13:04:03.206228 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.211094 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.213302 15156 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:03.218102 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.220467 15233 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:04:03.218748 15160 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.219017 15161 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.229194 15156 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:03.224026 18963 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:03.231448 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:03.232815 18772 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:03.233283 18772 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.233636 18772 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.237346 19001 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.238117 18801 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:03.238663 18801 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:03.244639 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.254980 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.255852 15156 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:03.239228 18826 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.257543 18801 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.216943 18750 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.264890 15160 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.216809 18827 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:03.268635 19000 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:04:03.269385 18929 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.269840 18867 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.270280 18750 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.268666 18801 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.285779 15088 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:03.191946 18612 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.288103 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.291456 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.292106 15158 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.272465 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.258430 18963 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.298804 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.294480 15160 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.293643 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.293171 18867 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:03.303325 18826 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:03.297991 18921 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:03.307683 18826 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.308209 18826 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.309594 19001 raft_consensus.cc:993] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 3d45101e0851484690b4f03a4dcdb63d to start an election
I20260709 13:04:03.310205 19001 raft_consensus.cc:1081] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Signalling peer 3d45101e0851484690b4f03a4dcdb63d to start an election
W20260709 13:04:03.312402 18846 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.003s
I20260709 13:04:03.318274 18840 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:03.305752 19006 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:04:03.320567 19006 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:03.321065 19006 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:03.321869 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.322817 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.328676 15161 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.327977 15081 raft_consensus.cc:2376] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8. Re-sending same reply.
W20260709 13:04:03.334049 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 72599.7us
I20260709 13:04:03.335052 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.323196 18938 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:03.299235 18963 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.331825 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.341611 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.338016 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.343693 18908 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:04:03.339502 18937 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:03.344674 19013 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:03.327715 18945 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:04:03.332976 18867 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.345270 19013 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:03.343945 15843 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:03.331898 18844 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:03.350914 15084 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:04:03.339609 18938 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:03.348410 18840 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:03.367993 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.370343 19001 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.335487 18946 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.358456 18827 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.384415 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.386722 15163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:57498
I20260709 13:04:03.387301 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 764) took 1071 ms. Trace:
I20260709 13:04:03.387427 15163 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.369632 18945 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:03.389173 15843 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:03.391139 15083 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:04:03.387524 19007 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:03.393564 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.394325 15079 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:03.389560 18946 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.387993 18945 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:03.397197 18945 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.397943 18976 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:03.402057 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.402906 15087 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.387835 15163 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:03.369410 18840 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:03.369949 18844 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:03.408732 18772 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.411625 19020 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.387674 15157 rpcz_store.cc:276] 0709 13:04:02.316165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.316341 (+   176us) service_pool.cc:224] Handling call
0709 13:04:02.316776 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.317650 (+   874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.317690 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:02.354322 (+ 36632us) log.cc:844] Serialized 52 byte log entry
0709 13:04:03.350132 (+995810us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b180004eb20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:03.370536 (+ 20404us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:03.384132 (+ 13596us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.385535 (+  1403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.385606 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.385643 (+    37us) raft_consensus.cc:1685] finished
0709 13:04:03.387095 (+  1452us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.387254 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7736,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":25177,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":7628,"spinlock_wait_cycles":5396224,"thread_start_us":33398,"threads_started":2,"wal-append.queue_time_us":998457}
I20260709 13:04:03.344695 18937 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:04:03.321960 19006 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:03.412388 19020 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.410346 18840 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.414268 18772 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.369410 18939 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:03.387501 18826 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.414855 15843 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:03.420230 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.415236 18844 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:03.425109 19003 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.222s	user 0.003s	sys 0.000s
I20260709 13:04:03.425660 18908 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.426257 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.433770 15081 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.439208 18945 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.427723 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.432330 15155 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:03.440317 15087 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:03.441943 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.442564 15087 raft_consensus.cc:2376] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8. Re-sending same reply.
I20260709 13:04:03.445225 15081 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:03.431962 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.452744 15085 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:03.451782 18838 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:03.392474 18938 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.429692 19001 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:03.475152 18684 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:03.475579 18684 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:03.454543 18908 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.476402 18684 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.466828 18838 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:03.493685 18838 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.495404 18838 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.498118 15230 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:03.501763 19020 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:03.505188 15848 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:03.505851 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1030) took 1551 ms. Trace:
I20260709 13:04:03.506129 15848 rpcz_store.cc:276] 0709 13:04:01.954186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.955377 (+  1191us) service_pool.cc:224] Handling call
0709 13:04:03.504804 (+1549427us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:04:03.504993 (+   189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.505628 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.505801 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2920064}
W20260709 13:04:03.513418 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52745.2us
I20260709 13:04:03.473485 18963 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.522744 18945 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.514236 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.430102 19013 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:03.520251 18979 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.269s	user 0.012s	sys 0.014s
I20260709 13:04:03.522171 18908 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.522054 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.552151 15237 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:04:03.486918 18938 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.521890 19001 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.530369 18684 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.535065 18826 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.540702 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.527622 15163 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.548947 15231 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:04:03.553664 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.555552 19028 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:03.557740 15234 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:03.558781 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.561054 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.561285 18963 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:04:03.585482 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44860.1us
I20260709 13:04:03.555495 18828 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:03.562489 18965 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:03.563663 15237 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:03.564052 18877 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.605433 15163 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.560559 19006 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:03.564606 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:03.565047 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.613175 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.569743 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.614953 15160 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.570272 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1880) took 2211 ms. Trace:
W20260709 13:04:03.580637 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.581306 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:03.551349 18937 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:03.582599 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.587801 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.589447 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.629566 15162 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.630885 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.632030 15229 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:03.590478 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.633257 15231 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:03.634644 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.635429 15231 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.606536 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.608006 15080 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:03.616055 15084 rpcz_store.cc:276] 0709 13:04:01.358631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.920858 (+1562227us) service_pool.cc:224] Handling call
0709 13:04:03.349200 (+428342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:03.360958 (+ 11758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.570104 (+209146us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400307350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:03.570221 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979913600}
I20260709 13:04:03.638119 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.624075 18877 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.639099 15229 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:03.640406 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.625379 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.641249 15232 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.641523 18877 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.642089 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.642225 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.642704 15229 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:04:03.643153 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.643411 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.597637 18828 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:03.643860 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.643942 15232 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.605400 18840 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.646482 15080 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:03.644548 15160 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.648770 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:03.648975 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54207.3us
W20260709 13:04:03.652546 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64864.7us
I20260709 13:04:03.599793 18908 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:03.653383 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.629931 18928 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:03.656033 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.656813 15229 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:04:03.657774 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.658448 15229 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.657663 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.662312 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.663040 18750 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.663218 15233 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:03.663654 18750 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:03.664601 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:03.667939 15233 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:03.668005 15160 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:03.676960 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.678558 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.669795 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.673125 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.680161 15080 raft_consensus.cc:2410] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:03.673573 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:03.664623 18750 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.668043 15162 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.655751 19026 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:03.654708 18937 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:03.692364 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.705641 15229 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:03.682577 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:03.706664 15232 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:03.706790 15235 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:03.713630 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.659175 18928 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:03.719316 15232 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:03.709481 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.721392 15232 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:04:03.723814 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 67546.9us
I20260709 13:04:03.682060 18822 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:04:03.727196 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.728849 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.731045 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.731463 15229 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:03.731792 18944 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.732376 18944 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.734012 18840 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:03.734513 18944 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:03.734825 15234 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:03.651973 18951 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:04:03.737659 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:03.739049 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85711.1us
I20260709 13:04:03.741586 15088 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.741935 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.740535 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.654356 18963 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:03.671226 18775 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.745204 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.671072 18908 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:03.747155 15234 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:03.758599 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:03.748224 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.748652 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.753572 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.687891 18923 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.763279 15084 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:03.765662 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.766218 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:03.683198 18892 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:03.767081 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.773576 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.774466 15159 raft_consensus.cc:2410] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:03.777415 15160 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.779038 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:03.779644 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.779954 15082 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:03.644680 18828 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.789340 15234 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:04:03.666213 18959 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:04:03.715690 18965 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.788635 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.714489 18616 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:03.790788 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:03.799615 15085 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:04:03.800051 15085 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:04:03.800256 15085 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:04:03.746501 18840 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.713115 18945 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:04:03.800477 15085 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:03.781062 19020 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:04:03.794557 15088 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:03.715269 18378 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:04:03.764919 18868 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:03.778680 18900 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:03.829869 15162 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:03.809720 15082 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:03.810943 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:03.803337 18616 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.753489 19026 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:04:03.803942 18965 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.817189 15084 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:03.814611 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1887) took 2213 ms. Trace:
I20260709 13:04:03.839000 15086 rpcz_store.cc:276] 0709 13:04:01.597273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.669899 (+ 72626us) service_pool.cc:224] Handling call
0709 13:04:01.670444 (+   545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.672557 (+1002113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.810819 (+1138262us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54000992d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:03.811041 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2288706432}
I20260709 13:04:03.827988 15231 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.832393 15081 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:03.841936 15081 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:03.834908 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.758162 18942 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:03.843012 15081 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:03.808179 18844 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.764364 18775 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.830819 18378 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:03.922961 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.817328 18922 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:03.979372 18922 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:03.908085 15235 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:03.814868 18780 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.812371 18986 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:03.905210 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.910866 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.837767 18944 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:03.847510 18900 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:03.811635 19037 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:03.906930 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.855496 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.905604 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:03.828734 18828 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:03.985510 15161 raft_consensus.cc:2368] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:03.818269 18908 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.980535 18922 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.988116 18922 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.906324 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.013737 15156 raft_consensus.cc:2368] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12.
I20260709 13:04:03.819463 18868 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 13:04:03.905977 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:04.017437 15155 raft_consensus.cc:2368] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:03.980137 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:03.989359 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 178997us
W20260709 13:04:03.859608 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.018563 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:03.916941 19042 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:04:04.031536 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.031898 15158 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:03.983614 18900 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:03.993340 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:03.994304 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:03.818120 18937 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.053857 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.846561 18822 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.025887 18838 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.028403 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.981658 18986 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:04.030050 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.033264 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.033615 15159 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:04.034562 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:04.035257 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.035372 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.039245 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:04.012310 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.046329 15158 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:04.064005 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:03.980759 18775 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:04.048491 15841 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:04.069254 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:04.070242 15839 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:04.070647 15844 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:04.044822 19039 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:04.074934 18838 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:04.076298 18938 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 13:04:04.076543 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:03.980175 18378 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.077374 18838 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.078392 15156 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:04:04.078758 15156 pending_rounds.cc:85] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 7
I20260709 13:04:04.078924 18378 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:04.079535 19041 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:04:04.079725 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.081915 15237 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:04.079017 15156 pending_rounds.cc:107] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:04.085417 15156 raft_consensus.cc:2889] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:04.088362 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49218.9us
I20260709 13:04:04.014407 18780 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:04.090453 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:04.012943 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 251635us
I20260709 13:04:04.093478 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.094307 15160 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:04.095206 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.095969 15162 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:04.095854 18772 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:04.038111 18844 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:04.072551 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:04.102829 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.107023 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:04.107687 15844 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:04.106175 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:04.105413 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:04.122851 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.128986 18986 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.130484 18986 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:04.132731 18827 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:04.133774 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.134963 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.135205 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.135782 15157 raft_consensus.cc:2410] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:04.136050 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.136288 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.137465 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:04.108788 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.110759 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:04.146960 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.172457 15158 raft_consensus.cc:2410] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:04:04.141503 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44061us
I20260709 13:04:04.091086 18616 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:04.175465 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 409131us
I20260709 13:04:04.101316 18772 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:04.187172 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 81953.5us
W20260709 13:04:04.189342 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.087308 18822 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.222642 15156 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:04.217257 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.063808 19044 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:04:04.108467 18963 consensus_queue.cc:1243] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:04.233232 15839 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:04.174995 19043 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.103214 18944 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 13:04:04.244443 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:04.247279 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 73804.9us
I20260709 13:04:04.210476 18772 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:04.249079 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.248242 19043 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:04.254580 15156 raft_consensus.cc:1217] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 13:04:04.255199 15156 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:04:04.109043 19051 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.141s
I20260709 13:04:04.258481 19043 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:04.254747 18892 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:04.272097 19001 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.271739 18844 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:04.274612 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 98167.7us
I20260709 13:04:04.275094 15840 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:04.246238 19039 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:04.247685 19054 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:04.323725 15159 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:04.324747 19047 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.254158 19037 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:04:04.276577 18780 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.413393 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:04.397926 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.414781 15088 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:04.304432 18963 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:04.403333 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 312949us
I20260709 13:04:04.418826 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.421334 15085 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:04.424875 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.311410 18965 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.408176 18750 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.443193 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:04.443437 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:04.471517 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.301385 18840 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:04.307583 19053 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.467366 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.418174 18963 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.469511 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.472734 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.490651 15237 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:04.491750 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.492568 15237 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:04.470909 15084 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:04.477618 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:04.478652 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:04.493592 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.494714 15159 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:04.494853 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:04.480005 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:04.480314 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61480.8us
I20260709 13:04:04.468004 15155 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:04.482959 15231 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:04.495518 15237 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:04.445353 19006 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.780s	user 0.006s	sys 0.005s
I20260709 13:04:04.412931 18892 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.426028 19048 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.502884 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48822.9us
I20260709 13:04:04.496798 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:04.498646 15232 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:04.389453 19039 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.540689 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44024.8us
I20260709 13:04:04.541869 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.530373 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.319713 19001 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.474157 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:04.536079 15080 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:04.544996 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:04.542723 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.545249 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:04.546308 15231 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:04.492599 18963 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:04.564041 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.565305 15232 raft_consensus.cc:2410] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:04.554003 15229 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:04.567437 15229 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:04:04.538606 18922 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.004s
I20260709 13:04:04.578841 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.585803 15237 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:04.607585 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:04.560096 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59330.2us
W20260709 13:04:04.609026 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.609422 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 67688.8us
I20260709 13:04:04.610373 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:04.609030 15231 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:04.418493 19047 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:04.534618 19066 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:04.617990 15079 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:04.491626 18684 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:04:04.485164 18750 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:04.626633 15157 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:04.627712 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.626395 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:04.634903 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:04.647575 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.649221 15844 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.003s
I20260709 13:04:04.655092 15844 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:04:04.652639 18867 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:04.625111 19061 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:04.633239 19052 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:04:04.649438 19026 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:04.626888 19066 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:04.646600 19074 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:04.581239 18965 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:04:04.648669 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.648033 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.553586 19039 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.633709 18922 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:04.672767 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.670964 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 128295us
I20260709 13:04:04.645824 18822 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.545662 19048 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.679044 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:04.679894 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.662479 18750 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:04.493151 19053 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.687209 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.662422 19061 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:04.663265 19066 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:04.663039 19026 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:04.681365 19006 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:04.662909 19077 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:04.692425 18923 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.907s	user 0.013s	sys 0.000s
I20260709 13:04:04.685843 19013 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:04.702389 15235 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:04.686599 18965 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:04.691860 18822 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:04.695829 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.636899 18840 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:04.688931 19044 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:04.748245 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 52513.7us
I20260709 13:04:04.702554 19026 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:04.656174 19069 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 13:04:04.757827 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77706us
I20260709 13:04:04.699193 19061 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:04.697007 19074 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:04.713410 18923 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:04.713521 18976 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.714147 19013 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.743414 18750 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:04.762007 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.723549 18965 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:04.773209 18976 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.774628 19013 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:04.779775 19058 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
I20260709 13:04:04.779965 19048 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:04.806725 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44703.1us
W20260709 13:04:04.782505 18828 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
I20260709 13:04:04.801966 15233 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:04:04.781137 19049 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 13:04:04.783481 19016 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 13:04:04.780781 19071 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:04:04.818174 19076 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:04:04.819672 18612 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:04.764213 19077 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:04:04.822901 15163 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:04.766222 18822 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:04:04.839521 19075 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:04:04.840435 18986 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.709s	user 0.000s	sys 0.012s
I20260709 13:04:04.842568 19054 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:04.809291 19001 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.829429 19053 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.834383 19061 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:04.831168 19068 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:04.811352 18750 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:04.828783 19078 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:04:04.851902 18976 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.860231 18963 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.820922 18945 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.849663 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1875) took 1814 ms. Trace:
W20260709 13:04:04.883826 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.844568 19046 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:04.873939 18986 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.874934 18750 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:04.850540 18772 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.003s
W20260709 13:04:04.891522 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 128777us
I20260709 13:04:04.883579 15228 rpcz_store.cc:276] 0709 13:04:03.035439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.035969 (+   530us) service_pool.cc:224] Handling call
0709 13:04:03.036364 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.045180 (+  8816us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:03.240991 (+195811us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.329621 (+ 88630us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.329728 (+   107us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.332731 (+  3003us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.332937 (+   206us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:03.335513 (+  2576us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.335762 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.335856 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.568316 (+232460us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18002b2400. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:04.612624 (+1044308us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b18002b2400. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:04.843474 (+230850us) raft_consensus.cc:1685] finished
0709 13:04:04.843497 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.849594 (+  6097us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":212363,"prepare.run_cpu_time_us":138,"prepare.run_wall_time_us":2711,"raft.queue_time_us":2197,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":17422,"spinlock_wait_cycles":14448896,"thread_start_us":251624,"threads_started":2,"wal-append.queue_time_us":102767}
I20260709 13:04:04.837458 19047 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:04.900063 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.904769 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:04.914697 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.148s	user 0.007s	sys 0.017s
W20260709 13:04:04.915223 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.008s	sys 0.017s
W20260709 13:04:04.915529 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.14929e+06us
I20260709 13:04:04.916332 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.890218 19039 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:04.919651 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:04.929541 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:04.929773 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.930442 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:04.930619 15237 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:04.930799 15234 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:04.931960 18891 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:04.932981 18976 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.934543 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:04.907413 18828 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.876152 19026 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:04.944938 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3309409170000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 13:04:04.895081 18986 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.849622 18965 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:04.932886 19082 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:04.955621 19052 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:04.957787 15126 connection.cc:256] RPC call timeout handler was delayed by -1.3201409170000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
I20260709 13:04:04.963335 15844 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:04.963785 15126 connection.cc:256] RPC call timeout handler was delayed by -1.0537409170000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:04:04.964955 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:04.965278 15126 leader_election.cc:400] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:04:04.965803 15126 leader_election.cc:403] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:04.892393 18945 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:04.945766 19039 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:04.973588 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:04.974203 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.925833 18909 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:04.960510 18937 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:04:04.982417 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:04.906487 18772 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.984198 15840 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:04.955456 18891 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:04.961155 19055 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:04:04.945861 18976 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:04.959514 18378 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 13:04:05.036928 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.022832 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.071218 15839 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:05.025005 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:05.020267 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2082) took 1371 ms. Trace:
I20260709 13:04:05.072299 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.073266 15154 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:04.926263 19044 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:04.947084 18684 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.072980 15233 rpcz_store.cc:276] 0709 13:04:03.648701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.678303 (+ 29602us) service_pool.cc:224] Handling call
0709 13:04:05.020093 (+1341790us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000587d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:05.020204 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445110656}
I20260709 13:04:05.082794 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.090898 15844 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:05.138520 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2620) took 1176 ms. Trace:
I20260709 13:04:05.139143 15154 rpcz_store.cc:276] 0709 13:04:03.904598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.906454 (+  1856us) service_pool.cc:224] Handling call
0709 13:04:05.024773 (+1118319us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:04:05.078105 (+ 53332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3226368}
I20260709 13:04:04.985970 18909 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:05.142299 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.151554 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:05.151890 15084 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:05.152221 15083 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:05.083737 15085 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:04.998485 18963 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:05.006857 18827 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.874s	user 0.013s	sys 0.000s
I20260709 13:04:05.149726 15086 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:04.968797 18780 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
I20260709 13:04:04.960907 18828 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.008503 18965 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.188766 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:05.190234 15052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.054s	sys 0.158s
W20260709 13:04:05.190546 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.054s	sys 0.158s
W20260709 13:04:05.190816 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 580516us
I20260709 13:04:05.148005 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:04.948634 19013 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.075495 18913 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.160389 15844 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:05.075224 19055 consensus_queue.cc:1243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:05.000720 18986 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.137663 18822 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.995230 19066 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:05.078733 19046 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.167261 19060 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.168283 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.169248 15236 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:05.237074 15236 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:05.237172 15084 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:05.238567 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.205035 18844 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:05.175457 19056 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.520s	user 0.000s	sys 0.003s
I20260709 13:04:05.186053 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:05.237926 15236 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:05.186174 19024 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:04:05.196081 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48216.1us
I20260709 13:04:04.998756 18945 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.154654 18909 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:05.202414 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:05.250823 15159 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:05.077677 18891 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.236515 18986 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:05.252697 18986 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:05.254676 18986 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:05.262883 19013 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:05.263041 19055 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.235586 19026 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.263579 19055 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.264145 18772 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.243707 18844 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.245067 18378 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.265118 18844 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.268051 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.205987 18913 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.023419 18951 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.190s
I20260709 13:04:05.225808 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.277173 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.278460 15844 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:04.991757 19088 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.206s
I20260709 13:04:05.137399 18684 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.190071 18911 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.184296 18827 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:05.239431 15084 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:05.171866 19006 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:05.249451 15236 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.232415 18822 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.265524 18891 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:05.302204 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.297016 15844 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:05.303503 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:05.275918 18945 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:05.305965 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:05.310586 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 363178us
I20260709 13:04:05.311124 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:05.312673 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.313410 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.283027 18772 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:05.301088 15236 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:04:05.314594 15236 pending_rounds.cc:85] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 4
I20260709 13:04:05.314855 15236 pending_rounds.cc:107] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:04:05.315125 15236 raft_consensus.cc:2889] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:05.249215 19060 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.326329 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1003) took 1375 ms. Trace:
I20260709 13:04:05.239080 19046 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:05.305260 18921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.409s	user 0.000s	sys 0.021s
I20260709 13:04:05.293151 19006 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:05.317466 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 352374us
I20260709 13:04:05.336239 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.329309 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2089) took 1654 ms. Trace:
I20260709 13:04:05.271296 18378 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:05.332806 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.327057 15848 rpcz_store.cc:276] 0709 13:04:03.950942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.951123 (+   181us) service_pool.cc:224] Handling call
0709 13:04:03.951554 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.952030 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.952060 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.952519 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.238336 (+285817us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:04.238391 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.238495 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.174681 (+936186us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:05.315901 (+141220us) raft_consensus.cc:1685] finished
0709 13:04:05.315924 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.326232 (+ 10308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":24138624,"thread_start_us":285367,"threads_started":1,"wal-append.queue_time_us":654488}
I20260709 13:04:05.341002 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.337338 15236 rpcz_store.cc:276] 0709 13:04:03.674617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.796494 (+121877us) service_pool.cc:224] Handling call
0709 13:04:03.796775 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.169139 (+1372364us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54000ca1a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:04:05.328932 (+159793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.329237 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008475008}
I20260709 13:04:05.317023 18772 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:05.334312 18921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.488s	user 0.000s	sys 0.022s
I20260709 13:04:05.351253 18772 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:05.351600 19092 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:05.343585 18976 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:05.357380 19091 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:04:05.359449 19060 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:05.360589 19089 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:05.361109 19102 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.361649 19102 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.362851 15087 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:05.364300 19102 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.369642 19088 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.380569 15084 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:05.301755 18911 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.293483 19090 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:04:05.391178 15845 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:05.385946 19095 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.385249 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 94277.8us
I20260709 13:04:05.389374 18684 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.378830 19102 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:04:05.396992 18951 consensus_queue.cc:1243] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:05.414705 18793 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:05.415858 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1895) took 1633 ms. Trace:
I20260709 13:04:05.425401 15082 rpcz_store.cc:276] 0709 13:04:03.782921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.783042 (+   121us) service_pool.cc:224] Handling call
0709 13:04:03.809400 (+ 26358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.831940 (+ 22540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.415698 (+1583758us) spinlock_profiling.cc:243] Waited 7.56 ms on lock 0x7b540034a650. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:05.415796 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17392000}
I20260709 13:04:05.428396 15082 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
W20260709 13:04:05.416682 15080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1890) took 1686 ms (client timeout 1741 ms). Trace:
W20260709 13:04:05.429534 15080 rpcz_store.cc:269] 0709 13:04:03.730600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.730721 (+   121us) service_pool.cc:224] Handling call
0709 13:04:05.416498 (+1685777us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:05.416626 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836044544}
I20260709 13:04:05.431209 15080 raft_consensus.cc:1217] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 13:04:05.431825 15080 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:04:05.422750 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.416689 15840 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:05.378698 18922 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:04:05.433995 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:05.420186 18909 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.424728 18684 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:05.413863 19055 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:05.456228 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.457857 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.429208 18793 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:05.428987 18928 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.413357 19046 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.437541 19074 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:05.442960 19066 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.443130 19098 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.437279 19053 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.467916 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.471220 15844 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:05.392850 19095 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.443666 18963 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:05.464977 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.453715 19104 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:05.458737 18684 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:05.477154 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1041) took 1483 ms. Trace:
I20260709 13:04:05.488080 15842 rpcz_store.cc:276] 0709 13:04:03.993621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.993737 (+   116us) service_pool.cc:224] Handling call
0709 13:04:05.470728 (+1476991us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54006cdc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:05.477060 (+  6332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593934208}
W20260709 13:04:05.473281 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73600.7us
I20260709 13:04:05.440527 18951 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:04:05.483492 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.491492 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.483515 18928 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:05.480728 19046 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.483733 19098 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:05.481452 18963 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.457834 18913 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:05.478368 15848 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:05.473047 19096 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:04:05.493126 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:05.482411 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 146269us
I20260709 13:04:05.487221 19006 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.499984 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:05.503153 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:05.510392 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 91516us
I20260709 13:04:05.509682 18963 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:05.520895 15201 connection.cc:256] RPC call timeout handler was delayed by -1.051876161999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 13:04:05.488194 19053 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.469096 19105 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:04:05.515678 19098 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:05.517482 18921 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:05.511130 18822 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.517534 19107 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:05.532239 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64411.7us
I20260709 13:04:05.515733 19095 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.534548 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.515985 19006 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.536638 15842 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:04:05.538389 15087 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:05.521457 18378 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.509621 19046 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.490996 19104 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.529250 19090 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.533650 19098 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:05.552175 18378 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 13:04:05.562819 19100 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:04:05.562561 19098 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.543497 19105 consensus_queue.cc:1243] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:05.564152 18844 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.546221 19103 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:04:05.545238 19107 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.536196 18921 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.543704 18951 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:04:05.547210 18822 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:04:05.533026 18684 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:05.797652 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.596334 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.812770 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:05.800318 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.845093 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 285997us
I20260709 13:04:05.850598 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:05.631477 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:05.859145 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 228203us
I20260709 13:04:05.860260 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.860535 18822 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:05.861542 18822 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:05.862053 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.863004 18822 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:05.709883 18891 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.865015 19074 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:05.866127 19074 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:05.866518 19074 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:05.866951 18891 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.867384 19074 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.869385 19074 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.877496 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.879624 18822 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:05.880892 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:05.882712 18822 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:05.860217 19105 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:05.884742 19105 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:05.885943 19105 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:05.895430 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:05.733922 19106 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:04:05.833097 19059 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.178s	user 0.005s	sys 0.000s
W20260709 13:04:05.805440 19109 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:04:05.833698 19047 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.776456 18965 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
W20260709 13:04:05.904335 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 356713us
I20260709 13:04:05.819629 19116 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:05.907596 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.635501 19044 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:05.802944 19104 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.783875 19098 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.785764 19088 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:05.785418 18909 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.834345 19046 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:05.657085 19114 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:05.848474 19107 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.916141 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.576114 18378 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.332s
W20260709 13:04:05.918685 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.575139 19090 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.833998 18928 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:04:05.920966 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.633989 18772 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:05.912578 18684 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:05.932186 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.932507 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:05.919617 19116 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:05.932910 15083 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:04:05.920120 19103 consensus_queue.cc:1243] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:05.925632 19114 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.927650 18846 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10.
I20260709 13:04:05.944592 15082 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:05.930359 19117 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:05.942793 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:05.949399 15083 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:05.915239 19047 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:05.918126 19044 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:05.964569 18772 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:05.953258 19104 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:05.948987 19103 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.041038 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1038) took 2048 ms. Trace:
I20260709 13:04:05.962777 18921 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.041493 15843 rpcz_store.cc:276] 0709 13:04:03.992557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.992714 (+   157us) service_pool.cc:224] Handling call
0709 13:04:04.044914 (+ 52200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.065129 (+ 20215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.065173 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.065765 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.249982 (+184217us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:04.250018 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.250088 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.828470 (+1578382us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:05.966975 (+138505us) raft_consensus.cc:1685] finished
0709 13:04:05.966997 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.040944 (+ 73947us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58138,"raft.queue_time_us":42886,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":6204,"spinlock_wait_cycles":6797184,"thread_start_us":173894,"threads_started":1,"wal-append.queue_time_us":471823}
I20260709 13:04:05.947553 19114 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.968429 18963 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.103891 19123 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.041436 19116 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:06.119760 18844 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.134069 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:06.132041 19123 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:06.143066 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 210688us
I20260709 13:04:06.144013 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.145269 19126 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.145527 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.149335 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.152354 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.153666 19090 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.133525 18844 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.137338 19117 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:06.166828 15087 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:04:06.166837 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:06.169556 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.145383 19088 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:04:06.176178 15087 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:06.177742 19103 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:04:06.180023 18939 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:06.180447 19103 consensus_queue.cc:1243] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:06.180943 19123 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.181052 19103 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:06.154357 19090 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.181536 19044 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:06.168231 19117 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.181898 19090 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.182379 19117 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:06.183995 19090 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.185519 18963 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.186091 18963 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.186740 19074 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.186827 18921 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.187286 19074 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.187520 18921 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.190021 18921 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:06.188025 19124 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:06.197733 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:06.151053 18378 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:06.154701 19104 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:06.202693 19047 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.204391 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 958) took 1828 ms. Trace:
I20260709 13:04:06.204779 15086 rpcz_store.cc:276] 0709 13:04:04.376293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.418129 (+ 41836us) service_pool.cc:224] Handling call
0709 13:04:04.418472 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.163183 (+744711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.204128 (+1040945us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54001b03d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:06.204321 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681235328}
I20260709 13:04:06.172667 19126 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.186554 18963 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.193562 15236 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:06.213660 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.195008 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.218701 15234 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.218902 15236 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:06.196323 15229 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:04:06.199940 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.223673 15229 pending_rounds.cc:85] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 6
I20260709 13:04:06.224093 15229 pending_rounds.cc:107] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:04:06.224023 15237 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 6.
I20260709 13:04:06.224655 15229 raft_consensus.cc:2889] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.225167 15229 pending_rounds.cc:107] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:06.225456 15229 raft_consensus.cc:2889] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.225852 15237 raft_consensus.cc:1217] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-8.8]   Dedup: 4.6->[7.7-8.8]
I20260709 13:04:06.226619 15237 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [4.6-8.8]
I20260709 13:04:06.227265 15229 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:06.212095 15235 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:06.228097 15237 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:06.173935 19102 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.170876 18844 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.246001 19107 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.247331 18844 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.248103 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 781) took 1838 ms. Trace:
W20260709 13:04:06.247444 19079 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.492s	user 0.003s	sys 0.000s
I20260709 13:04:06.248644 15158 rpcz_store.cc:276] 0709 13:04:04.409467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.409622 (+   155us) service_pool.cc:224] Handling call
0709 13:04:04.409956 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.418096 (+  8140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.418134 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.462587 (+ 44453us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.812049 (+1349462us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b18002078e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:05.925543 (+113494us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:05.925803 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.925887 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.247793 (+321906us) raft_consensus.cc:1685] finished
0709 13:04:06.247815 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.248042 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11082,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":38541,"spinlock_wait_cycles":13486592,"thread_start_us":109399,"threads_started":1,"wal-append.queue_time_us":1606208}
W20260709 13:04:06.256446 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1035) took 3372 ms (client timeout 1728 ms). Trace:
W20260709 13:04:06.259522 15845 rpcz_store.cc:269] 0709 13:04:02.883701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.883809 (+   108us) service_pool.cc:224] Handling call
0709 13:04:06.256247 (+3372438us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54004a2754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:06.256370 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5748430720}
I20260709 13:04:06.261734 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.262930 15237 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:06.268146 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70496.5us
I20260709 13:04:06.272563 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.268924 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.272794 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.274691 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:06.275817 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.276767 15237 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:06.248191 19107 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.281809 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.267433 19105 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:06.280421 19133 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:06.287832 19074 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.288228 19134 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.265038 19114 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.288825 19134 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.289788 19105 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:06.293051 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.294174 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.294754 19134 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.282796 18846 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:06.278860 19102 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:06.296661 19102 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:06.297612 19102 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.300302 15237 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:06.301529 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.301690 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.302203 15229 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.302336 15236 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 6.
I20260709 13:04:06.284771 19126 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:06.288573 19133 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:06.303180 19128 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:06.304348 19133 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:06.314247 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.315078 15157 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:06.315157 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.315310 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.316123 15161 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:06.316285 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.317080 15160 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:06.317795 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.318387 15156 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:06.321911 15229 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:06.322052 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.322669 15236 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:06.322921 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.308485 19114 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.326386 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.329401 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.330039 15154 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:06.330715 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.331378 15162 raft_consensus.cc:2410] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:06.345011 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1036) took 1386 ms. Trace:
I20260709 13:04:06.345444 15839 rpcz_store.cc:276] 0709 13:04:04.958519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.958784 (+   265us) service_pool.cc:224] Handling call
0709 13:04:06.344826 (+1386042us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:06.344965 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205656320}
I20260709 13:04:06.346304 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:06.335629 19086 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:06.347405 19086 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:06.348752 18963 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.350806 15158 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 6: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:06.353420 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.355587 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.356900 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.365609 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.374437 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.376705 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.348356 19086 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:06.381863 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:06.384060 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.384950 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:06.387838 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:06.386883 18921 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.390887 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:06.367848 18963 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.372323 19047 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:06.396171 15163 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2609) took 2648 ms (client timeout 2094 ms). Trace:
W20260709 13:04:06.396644 15163 rpcz_store.cc:269] 0709 13:04:03.747655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.747924 (+   269us) service_pool.cc:224] Handling call
0709 13:04:06.395943 (+2648019us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003e1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:06.396065 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2437438848}
I20260709 13:04:06.398577 18730 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:06.399121 18730 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.389107 19074 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.394644 19037 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:06.402046 19043 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:06.402456 19043 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:06.402369 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:06.403473 19043 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:06.404587 19043 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:06.405083 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.408351 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:06.410877 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.394882 18921 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:05.371568 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:06.376159 19117 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.448822 19117 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:06.424750 19047 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.449561 19047 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.449934 19047 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.434743 18730 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:06.453352 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.442414 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 964) took 2028 ms. Trace:
I20260709 13:04:06.455765 15202 leader_election.cc:400] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:06.456100 15202 leader_election.cc:403] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:06.455979 15079 rpcz_store.cc:276] 0709 13:04:04.414465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.414612 (+   147us) service_pool.cc:224] Handling call
0709 13:04:04.421090 (+  6478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.624459 (+203369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.442168 (+1817709us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540034ab54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:06.442344 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441201408}
I20260709 13:04:06.424448 18684 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:06.458315 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 140591us
I20260709 13:04:06.452054 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:06.460671 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.461545 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.462148 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:06.462919 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:06.462997 15082 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:04:06.463584 15087 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.459260 18921 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:06.464428 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.465456 18921 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.465929 18921 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:06.466305 18963 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:06.466737 18963 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.467136 18963 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:06.467480 19054 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:06.456781 19074 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
W20260709 13:04:06.467567 19065 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.836s	user 0.000s	sys 0.003s
I20260709 13:04:06.465821 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:06.493938 15079 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:06.469362 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:06.494932 15082 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:06.486163 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:06.496209 15086 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:06.497959 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.469054 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.458931 18684 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.506866 15081 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:06.405539 19037 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.502044 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1009) took 1320 ms. Trace:
I20260709 13:04:06.510057 15844 rpcz_store.cc:276] 0709 13:04:05.181928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.304281 (+122353us) service_pool.cc:224] Handling call
0709 13:04:05.470929 (+166648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.481720 (+ 10791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.501857 (+1020137us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54006cdc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:06.501989 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361035648}
I20260709 13:04:06.514834 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:06.515950 15083 raft_consensus.cc:2410] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:06.493129 15087 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:06.505151 15084 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:06.520102 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.520722 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.523115 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.513804 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.511875 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.523977 15840 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:06.531201 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:06.497356 18840 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:06.484704 19146 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.519151 18868 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.535530 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.512025 18822 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:06.536660 15842 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:06.537374 15079 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:06.537879 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:06.541396 15081 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:04:06.559422 15081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1897) took 2762 ms (client timeout 1697 ms). Trace:
W20260709 13:04:06.559914 15081 rpcz_store.cc:269] 0709 13:04:03.797253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.797373 (+   120us) service_pool.cc:224] Handling call
0709 13:04:06.506360 (+2708987us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5800825ed0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9400aa 00007f0d3f9c2447 00007f0d3f9dafe7 00007f0d3f9dc7de 00007f0d3f9e827c 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c
0709 13:04:06.559340 (+ 52980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353296768}
W20260709 13:04:06.542018 19145 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:04:06.512246 18684 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.542536 18843 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:06.484972 19054 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:06.503687 19044 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.545090 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:06.553041 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.535707 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:06.545143 19074 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:06.532457 18730 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.530084 19128 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:06.574571 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.555919 18868 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.529517 18928 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:06.575896 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:06.556115 19098 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:06.552239 18840 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.531994 19133 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:04:06.684329 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.586941 18843 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.683610 18730 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.774503 19133 consensus_queue.cc:1243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:06.775190 18730 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:06.810421 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.556457 19146 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.696628 19074 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:06.740118 14974 thread.cc:641] dir /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.012s	sys 0.022s
I20260709 13:04:06.911558 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:06.724496 19098 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:06.801592 18843 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:06.938234 18868 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:06.678874 19044 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.918735 19144 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.939198 19095 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:06.924124 14974 thread.cc:608] dir /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.013s	sys 0.022s
I20260709 13:04:06.725492 18928 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.923730 19074 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.928118 19098 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:06.939697 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.731997 18840 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:06.938421 19047 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:06.954000 19095 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:07.013792 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:06.979130 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 434075us
W20260709 13:04:07.023043 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 470893us
I20260709 13:04:07.042829 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:07.061012 19095 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.062469 19095 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:07.066143 19146 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.069891 19148 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:07.070261 19148 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:07.070588 19148 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:07.071115 18963 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 13:04:07.071542 18963 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:06.948931 19054 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.083662 15845 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:06.955873 19144 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.057525 19037 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:07.106230 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.006s	sys 0.030s
W20260709 13:04:07.117802 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.006s	sys 0.030s
I20260709 13:04:07.073998 19117 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:07.129774 19160 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:04:07.141873 19160 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.142452 19160 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:07.145351 15155 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:07.145809 15155 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:07.146667 15155 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:07.148164 15155 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:07.131011 19117 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.169365 19160 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:07.210569 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.007s	sys 0.016s
W20260709 13:04:07.211314 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.016s
W20260709 13:04:07.212175 15051 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.022s	sys 0.002s
W20260709 13:04:07.212664 15051 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.022s	sys 0.002s
W20260709 13:04:07.213109 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 590084us
W20260709 13:04:07.196208 18959 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:04:07.221293 19160 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:07.214517 18684 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.007s
I20260709 13:04:07.233947 19133 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:07.234741 19133 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:07.243022 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1002) took 3293 ms. Trace:
W20260709 13:04:07.244792 19151 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:07.246795 15235 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:07.220338 18963 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.243448 15847 rpcz_store.cc:276] 0709 13:04:03.949736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.949914 (+   178us) service_pool.cc:224] Handling call
0709 13:04:03.950314 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.982943 (+ 32629us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:04.204736 (+221793us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.415601 (+210865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.415655 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:04.415705 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.237913 (+2822208us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b18003e1400. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419
0709 13:04:07.242762 (+  4849us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:07.242829 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.242961 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24948,"prepare.queue_time_us":480460,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7413,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8677,"spinlock_wait_cycles":80177664,"thread_start_us":406782,"threads_started":2,"wal-append.queue_time_us":495416,"wal-append.run_cpu_time_us":5193,"wal-append.run_wall_time_us":1181814}
I20260709 13:04:07.229039 19044 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.229599 18928 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.279974 15155 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:04:07.280521 15155 pending_rounds.cc:85] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 3
I20260709 13:04:07.280814 15155 pending_rounds.cc:107] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:04:07.281297 15155 raft_consensus.cc:2889] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.281692 15155 pending_rounds.cc:107] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:04:07.281960 15155 raft_consensus.cc:2889] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.242095 18684 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.295848 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2124) took 2094 ms. Trace:
W20260709 13:04:07.791747 18822 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.210s	user 0.014s	sys 0.001s
W20260709 13:04:07.801280 19095 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.003s
I20260709 13:04:07.809827 15155 rpcz_store.cc:276] 0709 13:04:05.201676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.201798 (+   122us) service_pool.cc:224] Handling call
0709 13:04:05.202070 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.145226 (+1943156us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54004f0720. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:04:07.295467 (+150241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.295761 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517206400}
I20260709 13:04:07.811363 18822 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:07.443333 15229 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:07.812482 19095 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:07.813021 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:07.804232 19174 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:07.814050 15235 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:07.512014 18928 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.813097 19095 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.816473 18928 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:07.777119 15237 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:07.817971 19095 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.818635 18928 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:07.819252 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.015s
W20260709 13:04:07.819643 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.001s	sys 0.015s
I20260709 13:04:07.277947 19166 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:07.683586 18840 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.657s	user 0.020s	sys 0.000s
W20260709 13:04:07.733403 19043 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.312s	user 0.008s	sys 0.004s
I20260709 13:04:07.452864 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 785) took 1053 ms. Trace:
I20260709 13:04:07.325838 19144 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:07.737257 18921 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.271s	user 0.004s	sys 0.008s
W20260709 13:04:07.453176 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 410414us
I20260709 13:04:07.828064 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:07.828758 19117 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.489761 19074 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.505582 19165 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:07.508490 18843 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.002s
I20260709 13:04:07.466699 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2110) took 1146 ms. Trace:
I20260709 13:04:07.275952 19047 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:07.835700 18793 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:07.803037 19098 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.792s	user 0.016s	sys 0.003s
I20260709 13:04:07.896530 18793 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:07.792676 19167 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 13:04:07.481770 19162 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
W20260709 13:04:07.861907 19159 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.002s	sys 0.021s
W20260709 13:04:07.315356 19158 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:04:07.851469 15053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.008s	sys 0.008s
I20260709 13:04:07.812288 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:07.877820 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 792) took 1456 ms (client timeout 1722 ms). Trace:
I20260709 13:04:07.847467 15162 rpcz_store.cc:276] 0709 13:04:06.399781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.404938 (+  5157us) service_pool.cc:224] Handling call
0709 13:04:06.425994 (+ 21056us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:06.431627 (+  5633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.431672 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:06.432421 (+   749us) log.cc:844] Serialized 52 byte log entry
0709 13:04:06.703973 (+271552us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:06.770644 (+ 66671us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.452411 (+681767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.452518 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.452578 (+    60us) raft_consensus.cc:1685] finished
0709 13:04:07.452597 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.452774 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256969,"prepare.queue_time_us":66416,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":376234,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":8469,"thread_start_us":659715,"threads_started":2,"wal-append.queue_time_us":685854}
W20260709 13:04:07.864519 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.888725 15236 rpcz_store.cc:276] 0709 13:04:06.319864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.323198 (+  3334us) service_pool.cc:224] Handling call
0709 13:04:06.323440 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.323616 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:06.577259 (+253643us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.442847 (+865588us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.442907 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:07.442958 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.443038 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.466610 (+ 23572us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5113,"prepare.queue_time_us":818031,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":157647,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":11716,"thread_start_us":1112739,"threads_started":3,"wal-append.queue_time_us":644491}
I20260709 13:04:07.840195 18928 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:07.896713 19098 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.336726 18928 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:08.339432 19098 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.877822 18921 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.341346 18938 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.341526 19175 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:08.342463 18938 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.342622 19175 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.076696 19169 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.851265 19148 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.002s
I20260709 13:04:08.345183 19169 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:08.345567 19148 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:07.898483 18843 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.010627 19128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.109s	user 0.018s	sys 0.066s
I20260709 13:04:08.112648 19179 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.349558 19128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.618s	user 0.032s	sys 0.090s
I20260709 13:04:08.349445 19176 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:08.350098 19128 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:08.349973 19179 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.350668 18843 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.350471 19176 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:07.839913 19133 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:07.899960 19165 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:08.354900 19172 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.354965 18938 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.356034 19165 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:08.055462 19054 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.332995 19173 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.363898 18921 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.344125 18684 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.355849 19172 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.362331 19054 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.244278 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.009s
W20260709 13:04:08.364079 15814 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.003s	sys 0.007s
I20260709 13:04:08.110124 19170 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.274170 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.012s	sys 0.021s
W20260709 13:04:08.282011 15129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.007s	sys 0.003s
I20260709 13:04:08.351931 19128 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.281816 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.011s	sys 0.018s
I20260709 13:04:08.348651 19169 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:08.383944 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.011s	sys 0.032s
W20260709 13:04:08.384553 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 855979us
W20260709 13:04:08.375000 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.036s
W20260709 13:04:08.382169 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 793) took 1960 ms (client timeout 1535 ms). Trace:
W20260709 13:04:08.379189 15156 rpcz_store.cc:269] 0709 13:04:06.421600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.827703 (+1406103us) service_pool.cc:224] Handling call
0709 13:04:07.877732 (+ 50029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:07.862716 19043 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.899266 19047 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.394147 19047 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.395072 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:07.852099 18840 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.365139 18928 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:08.383708 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 794) took 1961 ms (client timeout 1809 ms). Trace:
W20260709 13:04:08.384691 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 527564us
W20260709 13:04:08.379590 15129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.011s	sys 0.010s
I20260709 13:04:07.847008 19166 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.363643 19173 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.379781 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.017s	sys 0.028s
I20260709 13:04:08.385615 19170 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.383477 19172 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:07.899087 19178 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.501s
I20260709 13:04:08.366004 19148 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.255285 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:08.639286 15085 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:08.638756 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2101) took 2364 ms. Trace:
I20260709 13:04:08.367340 18938 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:08.112998 19168 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:07.897779 19171 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.649168 15162 rpcz_store.cc:269] 0709 13:04:06.422705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.379447 (+1956742us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:08.383499 (+  4052us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:04:08.533591 15203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.127s (SENT) [suppressed 20 similar messages]
I20260709 13:04:08.471361 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1894) took 2151 ms. Trace:
I20260709 13:04:08.486866 15842 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:08.659929 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1900) took 2316 ms. Trace:
W20260709 13:04:08.653086 15053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.016s	sys 0.012s
I20260709 13:04:08.662593 18963 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:08.663281 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1905) took 3351 ms. Trace:
I20260709 13:04:08.459936 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:08.499091 18828 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.399385 18840 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.398636 19175 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.660398 15234 rpcz_store.cc:276] 0709 13:04:06.271790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.271986 (+   196us) service_pool.cc:224] Handling call
0709 13:04:06.272670 (+   684us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:06.273377 (+   707us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:06.734410 (+461033us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.583484 (+849074us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.714577 (+131093us) log.cc:844] Serialized 598 byte log entry
0709 13:04:07.714983 (+   406us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:07.715058 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.715198 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.463335 (+748137us) raft_consensus.cc:1685] finished
0709 13:04:08.463384 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.530730 (+ 67346us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b48002610d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:04:08.636442 (+105712us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":868015,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":113338,"raft.run_cpu_time_us":19140,"raft.run_wall_time_us":1127162,"spinlock_wait_cycles":2939008,"thread_start_us":991089,"threads_started":3,"wal-append.queue_time_us":391666}
I20260709 13:04:08.463644 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:08.473150 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:08.713429 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 799) took 2157 ms (client timeout 1934 ms). Trace:
I20260709 13:04:08.713958 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1064) took 2177 ms. Trace:
W20260709 13:04:08.714433 15157 rpcz_store.cc:269] 0709 13:04:06.556259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.459279 (+1903020us) service_pool.cc:224] Handling call
0709 13:04:08.713321 (+254042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.514962 18951 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:08.461722 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:08.519829 18976 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:04:08.714622 15842 rpcz_store.cc:276] 0709 13:04:06.536543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.485181 (+1948638us) service_pool.cc:224] Handling call
0709 13:04:08.486028 (+   847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.713611 (+227583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.713853 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.492189 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1010) took 3298 ms. Trace:
I20260709 13:04:08.525007 19006 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.453975 19159 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.011s	sys 0.022s
I20260709 13:04:08.721626 15156 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:08.716850 15229 rpcz_store.cc:276] 0709 13:04:06.320109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.323848 (+  3739us) service_pool.cc:224] Handling call
0709 13:04:07.443151 (+1119303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.812571 (+369420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.470846 (+658275us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540005bc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:08.471149 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573573504}
I20260709 13:04:08.480993 19144 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:08.723839 15231 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:04:08.386312 15155 rpcz_store.cc:269] 0709 13:04:06.422162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.811944 (+1389782us) service_pool.cc:224] Handling call
0709 13:04:08.381847 (+569903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.718297 15848 rpcz_store.cc:276] 0709 13:04:05.194133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.345983 (+151850us) service_pool.cc:224] Handling call
0709 13:04:05.353083 (+  7100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.510722 (+157639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.491563 (+2980841us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:08.491962 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287574912}
I20260709 13:04:08.713529 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:08.533239 19053 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.724974 15158 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:04:08.725880 19054 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.717545 18951 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:08.729153 19043 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:08.730165 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.019s
I20260709 13:04:08.730603 19116 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:08.732683 18929 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:08.653007 19176 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:08.737313 19118 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:04:08.744236 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2092) took 3810 ms (client timeout 1631 ms). Trace:
W20260709 13:04:08.739269 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2654) took 2327 ms (client timeout 2127 ms). Trace:
I20260709 13:04:08.735503 19144 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.735503 19006 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:08.746452 15810 connection.cc:256] RPC call timeout handler was delayed by -1.7054783349999525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
I20260709 13:04:08.741956 18929 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:08.747295 19030 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:08.747577 15159 raft_consensus.cc:2410] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:04:08.747694 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.749132 19026 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.748680 19038 raft_consensus.cc:670] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:04:08.746510 19043 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:08.742481 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.008s	sys 0.020s
W20260709 13:04:08.751091 15158 rpcz_store.cc:269] 0709 13:04:06.407302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.461035 (+2053733us) service_pool.cc:224] Handling call
0709 13:04:08.734720 (+273685us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:08.752578 15810 connection.cc:256] RPC call timeout handler was delayed by -1.5496055959999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:04:08.761075 15162 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:08.764058 15810 connection.cc:256] RPC call timeout handler was delayed by -1.5200783349999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:04:08.400084 19074 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:08.755254 19149 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.831s	user 0.009s	sys 0.011s
I20260709 13:04:08.765501 19030 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.766988 19054 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.767835 19054 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.756208 19152 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.555s	user 0.013s	sys 0.000s
I20260709 13:04:08.752199 19118 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:08.756467 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:08.750942 15231 rpcz_store.cc:269] 0709 13:04:04.929131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.931343 (+  2212us) service_pool.cc:224] Handling call
0709 13:04:08.472625 (+3541282us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:04:08.739820 (+267195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13580160}
I20260709 13:04:08.756440 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:08.717301 15232 rpcz_store.cc:276] 0709 13:04:06.343191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.138874 (+795683us) service_pool.cc:224] Handling call
0709 13:04:07.139219 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.453565 (+314346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.453607 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.454333 (+   726us) log.cc:844] Serialized 52 byte log entry
0709 13:04:07.862796 (+408463us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.622120 (+759324us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.646801 (+ 24681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.652666 (+  5865us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.652727 (+    61us) raft_consensus.cc:1685] finished
0709 13:04:08.652740 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.659018 (+  6278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74184,"prepare.queue_time_us":519352,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":144508,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":26260,"thread_start_us":776197,"threads_started":3,"wal-append.queue_time_us":1002089}
I20260709 13:04:08.756912 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:08.765762 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:08.762238 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 790) took 2355 ms (client timeout 2152 ms). Trace:
W20260709 13:04:08.769006 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2137) took 1322 ms (client timeout 1509 ms). Trace:
I20260709 13:04:08.765800 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:08.739610 18938 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.770018 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:08.916386 19197 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:08.916620 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 807) took 2723 ms. Trace:
I20260709 13:04:08.916970 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:08.917796 15158 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:08.744230 18951 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.915803 15088 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:08.864305 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.712852 15080 rpcz_store.cc:276] 0709 13:04:05.311822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.432630 (+120808us) service_pool.cc:224] Handling call
0709 13:04:05.434780 (+  2150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.435256 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.435299 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.435859 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.054785 (+1618926us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b18002ca220. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:07.258836 (+204051us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:07.279623 (+ 20787us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.501708 (+1222085us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18002ca220. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:08.662546 (+160838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.662684 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.662762 (+    78us) raft_consensus.cc:1685] finished
0709 13:04:08.662779 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.662997 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17233,"prepare.queue_time_us":20242,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":29088,"raft.queue_time_us":612374,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":96761,"spinlock_wait_cycles":35271680,"thread_start_us":336123,"threads_started":2,"wal-append.queue_time_us":1987667}
I20260709 13:04:08.917500 19026 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.947238 19054 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.770100 18929 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:08.949453 18929 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:08.951143 18822 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:08.917078 15233 rpcz_store.cc:276] 0709 13:04:06.193431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.193576 (+   145us) service_pool.cc:224] Handling call
0709 13:04:06.193972 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.197854 (+  3882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.197900 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:06.198483 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.476033 (+2277550us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b18003b96e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:08.886725 (+410692us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:08.886786 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.886931 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.916114 (+ 29183us) raft_consensus.cc:1685] finished
0709 13:04:08.916137 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.916453 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8311,"raft.queue_time_us":3505,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":4878,"spinlock_wait_cycles":10369792,"thread_start_us":275592,"threads_started":1,"wal-append.queue_time_us":2660038}
I20260709 13:04:08.952019 18822 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:08.924554 15159 rpcz_store.cc:269] 0709 13:04:06.406449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.463067 (+2056618us) service_pool.cc:224] Handling call
0709 13:04:08.762153 (+299086us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:08.954449 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 421037us
I20260709 13:04:08.770780 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:08.953756 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89932.4us
W20260709 13:04:08.955070 15162 rpcz_store.cc:269] 0709 13:04:07.443042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.713050 (+1270008us) service_pool.cc:224] Handling call
0709 13:04:08.764897 (+ 51847us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.947535 15844 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:08.917238 19197 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.967175 15848 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:08.967660 19038 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:08.897585 19047 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:08.845829 19030 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:08.972329 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 717524us
I20260709 13:04:08.965651 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:08.973616 15840 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:08.864746 19114 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.915417 19146 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.960254 18828 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.960497 18976 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:08.977810 15839 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:08.983228 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2135) took 1576 ms (client timeout 1777 ms). Trace:
I20260709 13:04:08.886806 19123 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:08.985595 15155 rpcz_store.cc:269] 0709 13:04:07.407012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.763496 (+1356484us) service_pool.cc:224] Handling call
0709 13:04:08.983175 (+219679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.975672 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:08.986892 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.995872 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:08.897174 18963 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.975092 19168 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:08.998498 15088 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:08.946336 19118 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.024946 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:09.020052 15157 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:09.031325 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 791) took 2613 ms (client timeout 2653 ms). Trace:
W20260709 13:04:09.031811 15157 rpcz_store.cc:269] 0709 13:04:06.418413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.723182 (+2304769us) service_pool.cc:224] Handling call
0709 13:04:09.031235 (+308053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:08.971145 19171 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.036725 15079 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.042862 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.042973 19197 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.045552 19197 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:09.049145 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.049489 18951 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:09.050021 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.051123 15844 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:04:09.032531 19146 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.053639 19026 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.029155 19114 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.054574 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:09.022967 18938 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.068593 15844 pending_rounds.cc:85] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 8
I20260709 13:04:09.055655 19144 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.034272 18828 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.056624 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:09.081833 15847 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.057170 15157 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:09.063570 15845 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:09.055900 15846 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:09.066200 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.986955 19173 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.068885 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:09.078584 15844 pending_rounds.cc:107] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:09.054473 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:09.075567 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:09.076187 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83428.5us
I20260709 13:04:09.003914 19038 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:09.096436 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.087168 15844 raft_consensus.cc:2889] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:08.999296 18976 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.107364 19197 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:04:09.109192 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:09.087002 19123 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.109684 19197 consensus_queue.cc:1243] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:09.110163 19123 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.111045 18976 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.107656 19206 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:09.114866 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:09.115940 19180 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:09.116215 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:09.117121 19196 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:04:09.098282 15159 raft_consensus.cc:2368] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:09.109311 15839 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:09.117872 19084 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:04:09.120993 15085 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:09.122483 19083 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:09.123415 15844 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:09.124953 15844 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:09.125128 19118 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.125453 15844 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:09.127097 19146 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.126891 15844 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.128360 15844 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.130026 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.131001 15159 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:09.135715 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.136224 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:09.136586 15233 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:09.132282 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.136951 15229 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:09.137360 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.138252 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.142573 18911 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.144368 19171 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.145198 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:09.146806 18828 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.149487 15232 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:09.149951 19168 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.150337 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:09.150712 19170 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:09.151731 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 789) took 2751 ms (client timeout 2011 ms). Trace:
I20260709 13:04:09.151747 15159 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:09.152523 19114 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.153321 19006 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.155202 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.156190 15085 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:09.156572 15081 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:09.157140 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.158540 15087 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:09.159418 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.160221 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.160965 15083 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:09.161158 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.161751 18976 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.161940 15084 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:09.162436 18976 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.164391 18822 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.159348 15161 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:09.166014 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 788) took 2765 ms. Trace:
I20260709 13:04:09.166399 15161 rpcz_store.cc:276] 0709 13:04:06.400533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.158552 (+2758019us) service_pool.cc:224] Handling call
0709 13:04:09.158899 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.165733 (+  6834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.165940 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.168093 15161 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:09.164746 18976 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.168722 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 795) took 2745 ms. Trace:
I20260709 13:04:09.169803 15080 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:09.169893 15161 rpcz_store.cc:276] 0709 13:04:06.423181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.167460 (+2744279us) service_pool.cc:224] Handling call
0709 13:04:09.167881 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.168478 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.168647 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.172300 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 796) took 2631 ms. Trace:
I20260709 13:04:09.172632 15161 rpcz_store.cc:276] 0709 13:04:06.541087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.171436 (+2630349us) service_pool.cc:224] Handling call
0709 13:04:09.171754 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.171949 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.171972 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.172027 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:09.172076 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.172138 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.172226 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.174261 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:09.174994 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 797) took 2632 ms. Trace:
I20260709 13:04:09.175413 15237 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:09.175474 15161 rpcz_store.cc:276] 0709 13:04:06.542210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.174031 (+2631821us) service_pool.cc:224] Handling call
0709 13:04:09.174404 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.174626 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.174657 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.174710 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:09.174751 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.174813 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.174939 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.179282 15233 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:09.180737 18822 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.180862 15155 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:09.181578 18822 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.182258 15155 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:09.182854 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 798) took 2628 ms. Trace:
I20260709 13:04:09.182965 15085 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:09.183277 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.183192 15155 rpcz_store.cc:276] 0709 13:04:06.554034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.181685 (+2627651us) service_pool.cc:224] Handling call
0709 13:04:09.182008 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.182636 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.182796 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.181990 15083 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:09.184492 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.184075 15087 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:09.125495 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:09.152230 15156 rpcz_store.cc:269] 0709 13:04:06.400758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.394304 (+1993546us) service_pool.cc:224] Handling call
0709 13:04:09.151500 (+757196us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540017dc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:09.151658 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743107840}
I20260709 13:04:09.193295 15162 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:09.196964 15079 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:09.198599 15847 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:09.199278 15088 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:09.200253 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.200204 15159 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:09.118948 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:09.212256 15846 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.213991 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 786) took 2807 ms. Trace:
I20260709 13:04:09.214496 15159 rpcz_store.cc:276] 0709 13:04:06.400061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.154022 (+2753961us) service_pool.cc:224] Handling call
0709 13:04:09.189423 (+ 35401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.206846 (+ 17423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.207108 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.217739 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.207234 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.219493 15158 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:04:09.220191 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2134) took 1815 ms (client timeout 1895 ms). Trace:
I20260709 13:04:09.185297 18822 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:09.220613 15158 rpcz_store.cc:269] 0709 13:04:07.405024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.839062 (+1434038us) service_pool.cc:224] Handling call
0709 13:04:09.220123 (+381061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.219509 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.223021 15156 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:09.112705 19123 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.225193 19095 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.227429 19038 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:09.229835 15231 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:09.245836 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 133219us
I20260709 13:04:09.246625 19006 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:09.247159 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:09.247293 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 129382us
I20260709 13:04:09.247903 19168 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:09.249992 19173 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.251034 18929 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.251644 15080 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:09.224647 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.261816 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2128) took 2003 ms. Trace:
I20260709 13:04:09.271374 15156 rpcz_store.cc:276] 0709 13:04:07.258044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.222164 (+1964120us) service_pool.cc:224] Handling call
0709 13:04:09.222566 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.252802 (+ 30236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.261736 (+  8934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.264600 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.272642 15158 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:04:09.273085 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 277592us
I20260709 13:04:09.261974 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.267395 19168 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.276950 15088 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:09.273850 15233 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:04:09.278752 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.279037 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.279476 15158 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:09.279420 19173 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:09.280153 15128 connection.cc:256] RPC call timeout handler was delayed by -1.5803815409999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
I20260709 13:04:09.280787 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:09.282399 19173 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.283103 18828 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.283304 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:09.284096 15228 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.284178 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:09.283134 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.286531 15087 pending_rounds.cc:85] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 6
I20260709 13:04:09.286947 15087 pending_rounds.cc:107] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:04:09.280673 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.287149 19038 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:09.287322 15087 raft_consensus.cc:2889] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.283797 18828 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.287617 15158 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:04:09.278767 19168 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.289891 18828 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.287617 19038 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.289038 19130 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:09.282358 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:09.288784 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.292506 15158 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:09.293838 15156 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:09.275686 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:09.300086 15844 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:04:09.302565 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.303143 15079 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:09.300633 15844 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 7
I20260709 13:04:09.305691 15844 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:04:09.305989 15844 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.291276 19038 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:09.319788 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.321947 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.320729 15156 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
I20260709 13:04:09.269914 19095 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.327454 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.330142 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.331259 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.331463 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.332048 15156 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:09.333326 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.334292 15156 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:09.335530 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:09.332158 15154 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:09.336333 19211 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:09.336886 19211 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.337195 15154 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:09.337297 19211 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.337867 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.338001 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2129) took 2079 ms. Trace:
I20260709 13:04:09.338670 15154 rpcz_store.cc:276] 0709 13:04:07.258364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.336480 (+2078116us) service_pool.cc:224] Handling call
0709 13:04:09.336950 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.337711 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.337943 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.341352 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.341535 15229 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:09.341706 15158 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:09.342471 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.342761 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.343410 15229 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:09.343577 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.344338 15233 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.344434 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.345347 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:09.344434 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62260.7us
I20260709 13:04:09.345986 15233 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.347189 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:09.348279 15235 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:09.349429 18938 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:09.349234 19069 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:09.349932 18938 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:09.350540 18938 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.350811 19114 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:09.351464 19114 consensus_queue.cc:1243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:09.351552 19069 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:09.275681 18929 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.353024 15236 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.353047 18822 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.353691 18822 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.354091 18822 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.356178 18929 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.356817 18929 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.357429 18929 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.357681 18911 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:09.358837 18911 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:09.359737 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2136) took 1942 ms. Trace:
I20260709 13:04:09.359889 18911 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:09.357448 15157 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:09.360965 18900 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:09.361245 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2133) took 2069 ms. Trace:
I20260709 13:04:09.361374 18900 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:09.361686 15157 rpcz_store.cc:276] 0709 13:04:07.291726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.356772 (+2065046us) service_pool.cc:224] Handling call
0709 13:04:09.357141 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.361012 (+  3871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.361193 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.362326 18900 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.363163 15157 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:09.363793 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2138) took 1835 ms. Trace:
I20260709 13:04:09.360368 18911 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:09.364180 15157 rpcz_store.cc:276] 0709 13:04:07.528108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.362507 (+1834399us) service_pool.cc:224] Handling call
0709 13:04:09.362859 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.363551 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.363733 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:09.365336 15201 connection.cc:256] RPC call timeout handler was delayed by -1.8710619800000359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:04:09.369022 15201 connection.cc:256] RPC call timeout handler was delayed by -1.8401677030001338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 13:04:09.369501 15201 connection.cc:256] RPC call timeout handler was delayed by -1.8127154270000574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 13:04:09.367422 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.328019 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:09.345719 15229 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.341507 15156 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.360142 15159 rpcz_store.cc:276] 0709 13:04:07.417717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.358940 (+1941223us) service_pool.cc:224] Handling call
0709 13:04:09.359252 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.359407 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.359431 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.359496 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:09.359535 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.359584 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.359671 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.366781 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.324385 19123 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:09.372256 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:09.414292 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100318us
I20260709 13:04:09.365190 18911 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.419445 18911 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.419651 18841 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:09.423575 19144 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:09.424032 19144 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:09.424670 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 114046us
I20260709 13:04:09.425031 19144 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.425544 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.426594 19144 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:09.427143 19026 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:09.427666 19026 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:09.428555 19026 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.429011 18844 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:04:09.426522 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:09.354120 19116 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:09.449105 19099 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 4.027s	user 0.008s	sys 0.036s
I20260709 13:04:09.429013 15234 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:09.449570 19206 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:09.415506 19047 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:09.460314 19118 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.455088 19061 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:09.457360 19166 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.457620 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:09.459514 18826 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:09.459893 19095 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:09.490533 15159 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:04:09.460592 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.435683 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2130) took 2169 ms (client timeout 2229 ms). Trace:
I20260709 13:04:09.461817 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2131) took 2140 ms. Trace:
W20260709 13:04:09.416561 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44421.7us
I20260709 13:04:09.442386 19179 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.493023 15154 rpcz_store.cc:276] 0709 13:04:07.277005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.339541 (+2062536us) service_pool.cc:224] Handling call
0709 13:04:09.339882 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.416615 (+ 76733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.416646 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.416698 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:09.416732 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.416792 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.416939 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.424064 15228 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:09.449545 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:09.328764 19173 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.483497 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57078.7us
I20260709 13:04:09.483331 18891 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:04:09.484205 18868 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:09.476269 19116 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.503093 19116 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.465660 19030 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.488817 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:09.492287 15162 rpcz_store.cc:269] 0709 13:04:07.266442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.963815 (+1697373us) service_pool.cc:224] Handling call
0709 13:04:09.435496 (+471681us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:09.435622 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317422976}
I20260709 13:04:09.492323 18900 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.497954 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.493703 19197 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:04:09.486377 19118 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.478609 19047 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.498935 19173 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.407724 19074 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.509639 19074 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.500702 18963 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:09.503996 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.493069 18841 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:09.510797 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:09.516288 15157 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:09.514070 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.521723 18963 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.521467 18841 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.459776 19006 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:09.522500 18963 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.523970 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.492314 18822 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.493321 19114 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:09.524616 15086 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.522097 14983 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.529814 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:09.506449 19030 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.510558 19047 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.508327 19118 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.516455 19054 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:09.511037 18929 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:09.566587 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50483.4us
I20260709 13:04:09.573154 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.571067 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.534657 19179 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.532195 19166 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.539502 19030 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.524802 18963 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:09.577476 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 119965us
I20260709 13:04:09.526561 18822 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.614220 19209 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:09.584560 19179 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.617158 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.632967 15080 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.584622 19144 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:04:09.618083 18929 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.618929 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.620280 18828 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:09.622526 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.623039 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.522135 19206 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:04:09.645264 15088 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:09.625191 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:09.635954 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 93213.5us
I20260709 13:04:09.646529 15083 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.625383 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.625837 14983 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:09.628369 15084 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:09.539088 18844 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:09.570500 19054 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:09.594192 19166 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.530462 18891 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:04:09.634744 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:09.634543 19179 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.649245 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.657833 18963 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:09.659848 18963 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.659955 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.660296 18963 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.661026 19096 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:09.645959 18828 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.646159 18929 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.662307 18929 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.661929 19096 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.657986 19166 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:09.664990 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 135363us
W20260709 13:04:09.654109 14981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 367) took 23345 ms (client timeout 15000 ms). Trace:
I20260709 13:04:09.669018 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1882) took 5186 ms. Trace:
I20260709 13:04:09.669425 15230 rpcz_store.cc:276] 0709 13:04:04.472127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.472225 (+    98us) service_pool.cc:224] Handling call
0709 13:04:04.472533 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.472885 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.472914 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.473340 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.022765 (+549425us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18003f3a60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:05.309348 (+286583us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:05.412284 (+102936us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.428872 (+ 16588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.429153 (+   281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.728890 (+3299737us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18003f3a60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db
0709 13:04:09.448358 (+719468us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:09.643852 (+195494us) raft_consensus.cc:1685] finished
0709 13:04:09.643872 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.658271 (+ 14399us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15999,"prepare.queue_time_us":7220,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":44816,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":31174,"spinlock_wait_cycles":38200704,"thread_start_us":378495,"threads_started":2,"wal-append.queue_time_us":926033}
I20260709 13:04:09.677039 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.677824 15082 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:09.657101 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.679960 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.661876 18828 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.661172 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.683420 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
W20260709 13:04:09.703166 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68521.8us
I20260709 13:04:09.703100 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.704607 15084 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:09.630638 19209 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.706632 14983 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 6, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 6 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.709498 14983 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:09.678575 18951 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.000s
I20260709 13:04:09.713660 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.714022 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:09.714713 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.715148 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.714310 15082 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.713707 15085 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:09.716681 14983 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 5, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 5 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.679319 19180 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:09.726759 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.724586 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 784) took 3249 ms. Trace:
I20260709 13:04:09.724786 15160 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:04:09.725322 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59616us
I20260709 13:04:09.740788 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:09.759666 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45980.4us
I20260709 13:04:09.760092 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.759181 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:09.770488 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.771448 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.721242 19209 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:09.656520 18844 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.786161 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.727679 15163 rpcz_store.cc:276] 0709 13:04:06.399479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.405645 (+  6166us) service_pool.cc:224] Handling call
0709 13:04:06.405890 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.406059 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:06.407776 (+  1717us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.157370 (+2749594us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b1800276160. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:04:09.576781 (+419411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.627190 (+ 50409us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:09.633236 (+  6046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.633322 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.648701 (+ 15379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2217,"prepare.queue_time_us":696,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":38417,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":48934,"spinlock_wait_cycles":23060736,"thread_start_us":374628,"threads_started":1}
I20260709 13:04:09.798190 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.798573 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.660174 19054 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:09.800083 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.769580 18822 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:09.813128 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.822501 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:09.823304 15088 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:09.824158 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.797713 19181 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:09.827632 15085 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:04:09.829710 15155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.011s	sys 0.006s
W20260709 13:04:09.830188 15155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.011s	sys 0.006s
W20260709 13:04:09.830876 15237 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.004s	sys 0.012s
W20260709 13:04:09.831398 15237 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.004s	sys 0.012s
I20260709 13:04:09.831445 19197 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:09.832031 19197 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:09.833081 19197 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.834632 19197 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.802256 19096 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:09.837230 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.758980 19211 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:09.821803 18822 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.850603 15156 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:09.838068 19074 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:09.847153 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 362158us
W20260709 13:04:09.898196 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 169855us
I20260709 13:04:09.871520 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.907627 15079 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:09.890936 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 441588us
W20260709 13:04:09.910241 15842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.014s	sys 0.005s
W20260709 13:04:09.871138 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 112044us
I20260709 13:04:09.907811 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.910075 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:09.667155 14981 rpcz_store.cc:269] 0709 13:03:46.278790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.289329 (+ 10539us) service_pool.cc:224] Handling call
0709 13:04:09.616709 (+23327380us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:57.173937 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:57.174822 (+   885us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:57.174876 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:03:57.174908 (+    32us) tablet.cc:662] Decoding operations
0709 13:03:57.197622 (+ 22714us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:57.197703 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:57.197733 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:03:57.198711 (+   978us) tablet.cc:701] Row locks acquired
0709 13:03:57.198724 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:03:57.198851 (+   127us) write_op.cc:270] Start()
0709 13:03:57.198935 (+    84us) write_op.cc:276] Timestamp: P: 1783602237198810 usec, L: 0
0709 13:03:57.198949 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 13:03:57.199522 (+   573us) log.cc:844] Serialized 7177 byte log entry
0709 13:03:58.797794 (+1598272us) op_driver.cc:464] REPLICATION: finished
0709 13:03:59.608663 (+810869us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b18002983c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:03:59.608885 (+   222us) write_op.cc:301] APPLY: starting
0709 13:03:59.609015 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:59.609495 (+   480us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:59.609530 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:59.611808 (+  2278us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:59.612179 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:03:59.612226 (+    47us) write_op.cc:312] APPLY: finished
0709 13:03:59.612729 (+   503us) log.cc:844] Serialized 125 byte log entry
0709 13:04:00.473090 (+860361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:00.561713 (+ 88623us) write_op.cc:454] Released schema lock
0709 13:04:00.561872 (+   159us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:00.562399 (+   527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:00.562435 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":608332,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":645896,"apply.run_cpu_time_us":40149,"apply.run_wall_time_us":953497,"num_ops":18,"prepare.queue_time_us":469975,"prepare.run_cpu_time_us":20387,"prepare.run_wall_time_us":352261,"raft.queue_time_us":781572,"raft.run_cpu_time_us":96554,"raft.run_wall_time_us":393489,"replication_time_us":1598625,"spinlock_wait_cycles":40239488,"thread_start_us":2306487,"threads_started":5,"wal-append.queue_time_us":1363049,"wal-append.run_cpu_time_us":26983,"wal-append.run_wall_time_us":2768671}]]}
I20260709 13:04:09.898816 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:09.900319 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 787) took 3489 ms. Trace:
I20260709 13:04:09.916311 15160 rpcz_store.cc:276] 0709 13:04:06.400293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.158113 (+2757820us) service_pool.cc:224] Handling call
0709 13:04:09.158508 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.727799 (+569291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.881829 (+154030us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54001c70d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:09.889313 (+  7484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289019136}
I20260709 13:04:09.923564 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:09.923736 15201 connection.cc:256] RPC call timeout handler was delayed by -1.6308705300000383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:04:09.928725 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:09.930339 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 308084us
I20260709 13:04:09.930657 15081 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:04:09.931295 15842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.015s	sys 0.006s
I20260709 13:04:09.871169 18822 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:09.929244 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:09.948743 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:09.900013 18868 consensus_queue.cc:1243] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:09.943980 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:09.898685 19054 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.964435 15088 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:09.953099 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 800) took 3325 ms. Trace:
I20260709 13:04:09.954018 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:09.934979 19160 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.448s	user 0.005s	sys 0.032s
I20260709 13:04:09.924202 19074 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:09.882035 19173 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:09.833104 19204 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
W20260709 13:04:09.978446 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.979806 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2132) took 2700 ms. Trace:
I20260709 13:04:09.996465 15158 rpcz_store.cc:276] 0709 13:04:07.279375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.342529 (+2063154us) service_pool.cc:224] Handling call
0709 13:04:09.342865 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.343066 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:09.680535 (+337469us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.979444 (+298909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.979505 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:09.979548 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.979611 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.979731 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":425403,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":16343,"thread_start_us":539903,"threads_started":2}
I20260709 13:04:09.965972 15161 rpcz_store.cc:276] 0709 13:04:06.628151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.184657 (+2556506us) service_pool.cc:224] Handling call
0709 13:04:09.185003 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.185304 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:09.463460 (+278156us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.952438 (+488978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.952563 (+   125us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:09.952687 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.952826 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.953026 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":492304,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3224,"raft.queue_time_us":12413,"thread_start_us":684040,"threads_started":2}
I20260709 13:04:09.982816 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:09.969173 19160 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.704s	user 0.006s	sys 0.033s
I20260709 13:04:09.969408 19074 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:09.967460 19205 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:09.985728 19173 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:10.013406 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 107699us
I20260709 13:04:10.006817 19160 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:10.017131 19227 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.017642 19227 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.018008 19227 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:10.018522 19218 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:10.018085 19160 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:10.021045 19220 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:04:10.021256 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.021206 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:10.012241 14983 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 5 to 6, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 6 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.982249 19144 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:04:10.177282 19166 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.512s	user 0.004s	sys 0.006s
I20260709 13:04:10.103117 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.104656 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:09.990291 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 230496us
I20260709 13:04:10.104214 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:10.189230 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.001319 18891 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:10.103785 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:10.142679 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89971.4us
I20260709 13:04:10.102305 14983 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 5, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 5 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:04:10.169822 18938 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.007s
I20260709 13:04:10.238157 15844 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 6.
W20260709 13:04:10.092887 18877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.020s
W20260709 13:04:10.171820 18963 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:04:10.092926 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:10.093293 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:10.108764 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.179148 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.016640 15086 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:10.009127 19074 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:10.162410 15082 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:10.029488 19211 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:10.248725 18877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.006s	sys 0.026s
W20260709 13:04:10.267192 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 298140us
I20260709 13:04:10.105293 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.052544 15083 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:10.069288 15080 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:10.103380 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.091773 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:10.090919 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2127) took 2843 ms. Trace:
W20260709 13:04:10.103335 19116 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.586s	user 0.004s	sys 0.017s
I20260709 13:04:10.001892 19234 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:10.100261 19006 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.004s
W20260709 13:04:10.108256 19213 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:04:10.061770 19168 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.772s	user 0.013s	sys 0.009s
I20260709 13:04:10.015472 19173 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.219425 15847 raft_consensus.cc:2368] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 7.
W20260709 13:04:10.095552 19038 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.000s
I20260709 13:04:10.049141 19102 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:10.248493 18900 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.737s	user 0.003s	sys 0.008s
I20260709 13:04:10.221622 15839 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:10.178069 19166 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.230696 18891 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:10.263748 14983 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:10.083117 19214 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.193s
W20260709 13:04:10.281670 15201 connection.cc:256] RPC call timeout handler was delayed by -1.5205290820000528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 13:04:10.259696 18963 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:10.303841 19030 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.671s	user 0.017s	sys 0.003s
I20260709 13:04:10.288439 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:10.031785 18951 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.235s
I20260709 13:04:10.091964 19118 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:10.291349 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.295217 15087 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:10.305445 19038 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.297602 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:10.291023 18929 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.006s
W20260709 13:04:10.323351 18828 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.604s	user 0.004s	sys 0.004s
I20260709 13:04:10.249627 18938 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.280871 18844 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.309113 15845 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:04:10.311160 15201 connection.cc:256] RPC call timeout handler was delayed by -1.4417619800000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 13:04:10.293286 19095 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:10.284129 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:10.288964 19234 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:10.308817 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 388152us
W20260709 13:04:10.314033 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 222386us
I20260709 13:04:10.300657 19173 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:10.313684 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.322129 19232 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:10.323331 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:10.301755 19102 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.285785 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:10.327780 19225 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:10.329804 19166 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.288650 19116 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.273989 15155 rpcz_store.cc:276] 0709 13:04:07.247959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.282498 (+2034539us) service_pool.cc:224] Handling call
0709 13:04:09.282908 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:09.283422 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.283463 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:09.284108 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 13:04:09.830685 (+546577us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:09.897958 (+ 67273us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.898228 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.898328 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.090720 (+192392us) raft_consensus.cc:1685] finished
0709 13:04:10.090737 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.090865 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27056,"prepare.queue_time_us":66267,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":4152,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":19675,"thread_start_us":546129,"threads_started":1,"wal-append.queue_time_us":693533}
I20260709 13:04:10.349210 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:10.422999 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 99906.5us
I20260709 13:04:10.349941 19206 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:10.351739 18911 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:10.432739 18911 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:10.433712 18911 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.434717 18911 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:10.436537 19096 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:10.436976 19095 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:10.437361 19095 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:10.437726 19095 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:10.439812 19170 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:10.440217 19170 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:10.441027 15159 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:10.442225 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 807) took 1392 ms. Trace:
I20260709 13:04:10.442570 15159 rpcz_store.cc:276] 0709 13:04:09.049495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.509384 (+459889us) service_pool.cc:224] Handling call
0709 13:04:09.509698 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.441449 (+931751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.441989 (+   540us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54003e0b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:10.442171 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141400960}
W20260709 13:04:10.359010 19179 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:04:10.444144 19179 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.367365 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1019) took 1393 ms. Trace:
W20260709 13:04:10.452704 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 203291us
I20260709 13:04:10.453755 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:10.454882 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:10.455113 19205 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:10.455777 19205 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:10.369831 18822 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:10.456743 19214 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:10.457239 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.395329 19232 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:10.458031 19232 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:10.459905 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.461236 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:10.462085 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.463238 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:10.464023 15154 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:10.375506 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:10.377168 14974 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.788s	user 0.004s	sys 0.028s
W20260709 13:04:10.377600 19209 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:04:10.106909 19026 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.181s
W20260709 13:04:10.381824 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.382944 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.298915 19168 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.300073 19235 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:04:10.388319 15201 connection.cc:256] RPC call timeout handler was delayed by -1.356261980000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 13:04:10.492012 15201 connection.cc:256] RPC call timeout handler was delayed by -1.263570360000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:04:10.391055 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:10.393862 19251 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.394179 19233 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:10.384917 18929 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:10.395854 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.390278 18828 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.398036 14983 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 2, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 2 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:10.427299 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.429467 19116 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.430724 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:10.431602 19255 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:10.432297 19206 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:10.465278 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:10.402036 19197 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.002s
I20260709 13:04:10.338567 19030 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.468974 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:10.488967 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.467039 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.420809 18891 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.534083 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:10.534091 18891 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:10.534834 19074 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:10.536221 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 105576us
I20260709 13:04:10.536901 19227 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.538884 19227 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.540475 19030 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:10.541481 15846 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:04:10.492553 15201 connection.cc:256] RPC call timeout handler was delayed by -1.1345215620000317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2413s for a response
I20260709 13:04:10.534207 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.515379 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:10.542328 15201 connection.cc:256] RPC call timeout handler was delayed by -1.0160619800000177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26080000000000003s for a response
I20260709 13:04:10.542918 15847 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:10.509703 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:10.515698 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.395331 18938 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.392712 19234 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.408113 19102 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.465597 15843 rpcz_store.cc:276] 0709 13:04:08.977278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.982351 (+  5073us) service_pool.cc:224] Handling call
0709 13:04:09.032921 (+ 50570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.104715 (+ 71794us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:09.594252 (+489537us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.947967 (+353715us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.957518 (+  9551us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.957688 (+   170us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:09.957732 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.957820 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.367039 (+409219us) raft_consensus.cc:1685] finished
0709 13:04:10.367067 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.367255 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120886,"prepare.queue_time_us":321903,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":121185,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":45518,"thread_start_us":730893,"threads_started":3,"wal-append.queue_time_us":744752}
W20260709 13:04:10.413096 19047 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.810s	user 0.005s	sys 0.016s
I20260709 13:04:10.550923 15157 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:10.551280 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.558718 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.554152 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.559828 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.561187 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.562430 15840 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:04:10.555511 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 86657.2us
I20260709 13:04:10.556245 15848 raft_consensus.cc:2368] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:10.554498 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:10.552559 15845 raft_consensus.cc:2368] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:10.535053 18828 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.532401 19173 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.517423 19205 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.565615 18822 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:10.566664 15088 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:10.530481 18951 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:10.578943 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.580302 19205 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:10.555204 19251 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:04:10.582495 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 72908.6us
I20260709 13:04:10.580535 18929 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:10.582798 19179 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.583109 19255 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:10.583801 19030 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.578991 19173 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.494431 19166 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:04:10.586170 19118 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.586517 19074 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:10.553505 19096 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.587518 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.555578 19116 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.625025 19116 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.625484 19116 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.565973 18826 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.596277 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:10.597412 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.599324 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.576190 14983 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 4, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:10.585642 19230 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:10.601336 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.604799 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.606148 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.632059 15844 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:10.567689 19254 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.609395 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.588393 18951 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:10.600960 18868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.084s	sys 0.081s
I20260709 13:04:10.493990 19209 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.618494 19179 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:04:10.632809 15848 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:10.600499 19227 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:10.648995 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52789.7us
I20260709 13:04:10.600580 19030 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:10.569128 19047 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.603055 18822 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:10.653053 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.655323 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.655406 19243 raft_consensus.cc:2764] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:04:10.655637 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:10.648550 19160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.029s	sys 0.061s
W20260709 13:04:10.656241 19248 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.091s	user 0.000s	sys 0.002s
I20260709 13:04:10.659017 19179 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:10.661362 19180 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:10.662096 18844 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:10.661500 14983 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 1 to 4, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 4 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:10.647439 18868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.085s	sys 0.081s
I20260709 13:04:10.618394 19206 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.664119 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.665925 19211 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:10.666059 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.587553 19181 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:04:10.619848 19096 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:10.558137 19170 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.596165 19255 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.709544 15841 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 6. Current term is 8. Ops: []
I20260709 13:04:10.791582 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.751868 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.817008 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:11.151554 15844 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:10.677191 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:10.747339 18963 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:10.647094 19254 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.646832 18826 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.164999 19254 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:11.165154 18826 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:10.855665 18877 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.009s	sys 0.013s
W20260709 13:04:10.855279 18976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.017s
I20260709 13:04:10.653532 19227 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:11.129624 19251 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:11.102751 19242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.033s	sys 0.074s
I20260709 13:04:10.673993 19030 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.816479 19038 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:11.124801 19253 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:04:11.149312 19160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.043s	sys 0.086s
I20260709 13:04:10.933956 19214 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:11.150087 19179 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:10.910169 19144 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.705873 19166 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:11.150610 18822 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:11.259961 18976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.007s	sys 0.035s
W20260709 13:04:11.223443 18877 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.014s	sys 0.019s
I20260709 13:04:11.160305 19096 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:11.311524 18822 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:11.259186 19170 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:11.272262 19253 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:11.314594 19206 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:10.709184 19117 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:10.672691 19095 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:10.945940 19232 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.201056 19209 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:11.257292 19047 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:11.261463 19242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.034s	sys 0.083s
I20260709 13:04:10.815059 18951 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.261607 19227 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:10.653898 19230 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.149283 19211 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:11.440595 18976 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.450404 19257 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:04:11.312405 19166 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.312623 19144 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.451815 19259 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:11.484069 19261 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.402s	user 0.000s	sys 0.001s
I20260709 13:04:11.397605 19179 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:11.531648 19265 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.634941 19116 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.009s	user 0.019s	sys 0.001s
I20260709 13:04:11.628703 19264 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:11.652597 18951 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:11.491187 19253 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.475299 19117 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.628865 19211 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.762699 19179 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:11.741624 19116 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.699939 18976 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.675783 19259 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:11.711717 19214 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:11.786589 19265 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:12.338796 19211 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:12.317780 19095 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:12.962344 19234 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 2.390s	user 0.009s	sys 0.004s
I20260709 13:04:11.778532 19264 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:12.995882 19232 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:12.996104 19144 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:13.002038 19234 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:13.034101 19274 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.638s	user 0.001s	sys 0.000s
I20260709 13:04:13.040489 19166 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:13.050822 19102 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 2.486s	user 0.007s	sys 0.007s
I20260709 13:04:13.057224 19116 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:13.062000 19102 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:13.073140 19102 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:13.074050 18827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.614s	user 0.008s	sys 0.009s
W20260709 13:04:13.074585 19030 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.804s	user 0.006s	sys 0.004s
I20260709 13:04:13.075034 19030 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:13.076464 18822 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.679s	user 0.006s	sys 0.002s
I20260709 13:04:13.060766 19230 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:13.064731 18976 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:13.085198 19117 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:13.083674 19265 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:13.069696 19259 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:13.082866 18822 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:13.094069 19254 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.928s	user 0.009s	sys 0.002s
I20260709 13:04:13.094565 19254 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:13.095126 18826 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.929s	user 0.010s	sys 0.002s
I20260709 13:04:13.095957 18826 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:13.096617 19254 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:13.074601 18827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.652s	user 0.009s	sys 0.009s
I20260709 13:04:13.117694 19030 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:13.418689 18822 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:13.420063 19096 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.979s	user 0.010s	sys 0.000s
I20260709 13:04:13.420521 19096 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:13.422711 19096 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:13.423537 19179 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.155s	user 0.006s	sys 0.002s
I20260709 13:04:13.410336 19234 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:13.420085 19227 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.786s	user 0.008s	sys 0.000s
I20260709 13:04:13.429653 19227 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:13.431605 19227 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:13.432128 19264 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:13.426296 19170 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 2.003s	user 0.010s	sys 0.001s
I20260709 13:04:13.438900 18826 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:13.431694 18951 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.653s	user 0.008s	sys 0.000s
W20260709 13:04:13.588893 19255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.016s	sys 0.031s
W20260709 13:04:14.289316 19255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.017s	sys 0.031s
W20260709 13:04:14.321651 19214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.036s	sys 0.055s
W20260709 13:04:14.339967 18827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.035s	sys 0.058s
W20260709 13:04:14.347494 18827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.035s	sys 0.058s
W20260709 13:04:14.347167 19214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.036s	sys 0.056s
W20260709 13:04:14.361513 19277 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.934s	user 0.002s	sys 0.000s
W20260709 13:04:14.366762 19211 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.382s	user 0.004s	sys 0.007s
W20260709 13:04:14.374768 19206 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 2.923s	user 0.013s	sys 0.001s
I20260709 13:04:14.421437 15154 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:10.689302 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.441776 15126 connection.cc:256] RPC call timeout handler was delayed by -3.687259612000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 13:04:14.442166 15126 connection.cc:256] RPC call timeout handler was delayed by -3.6785713059999807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:04:14.442487 15126 connection.cc:256] RPC call timeout handler was delayed by -3.6530596119999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 13:04:14.442919 15126 connection.cc:256] RPC call timeout handler was delayed by -3.5858713060000582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 13:04:14.443264 15126 connection.cc:256] RPC call timeout handler was delayed by -3.579910271000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 13:04:14.443032 19214 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:14.443670 15126 connection.cc:256] RPC call timeout handler was delayed by -3.4904713060000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:04:14.443949 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.444234 15126 connection.cc:256] RPC call timeout handler was delayed by -3.4058713059999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
I20260709 13:04:14.443815 19214 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:14.444497 15126 connection.cc:256] RPC call timeout handler was delayed by -3.29337130600004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:04:14.444941 15126 connection.cc:256] RPC call timeout handler was delayed by -3.1005531590000146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:04:14.445279 15126 connection.cc:256] RPC call timeout handler was delayed by -2.6456047670000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:04:14.445576 15126 connection.cc:256] RPC call timeout handler was delayed by -2.6417713060000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27330000000000004s for a response
W20260709 13:04:14.445660 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 4.05437e+06us
W20260709 13:04:14.445907 15126 connection.cc:256] RPC call timeout handler was delayed by -2.0571421840000994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 13:04:14.446190 19214 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:14.446988 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:14.448411 19280 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:14.448792 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:14.450196 19280 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:14.451484 19280 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.452442 19280 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:10.937551 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.457130 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.458093 19214 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:14.458174 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.458760 19214 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:14.459352 19214 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:14.461606 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 810) took 4007 ms (client timeout 1751 ms). Trace:
W20260709 13:04:14.465678 15159 rpcz_store.cc:269] 0709 13:04:10.454327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.472671 (+ 18344us) service_pool.cc:224] Handling call
0709 13:04:14.461274 (+3988603us) spinlock_profiling.cc:243] Waited 3.89 s on lock 0x7b540013ecd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:14.461477 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8957604608}
I20260709 13:04:14.467152 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:14.466073 19283 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:14.467964 19214 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.468945 19283 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:14.469594 19283 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.470553 19214 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.471201 19214 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:14.471589 19214 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.471946 19214 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.472064 19281 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.472709 19281 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.475023 19281 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.475562 19281 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:14.476415 15840 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:14.476189 19281 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.477941 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:14.480921 19214 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.481911 19281 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:14.482501 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1071) took 3988 ms (client timeout 1990 ms). Trace:
I20260709 13:04:14.483111 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:14.482882 15840 rpcz_store.cc:269] 0709 13:04:10.488989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.577003 (+ 88014us) service_pool.cc:224] Handling call
0709 13:04:14.477319 (+3900316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.489131 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1076) took 3886 ms. Trace:
I20260709 13:04:14.492468 15840 rpcz_store.cc:276] 0709 13:04:10.600344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.484572 (+3884228us) service_pool.cc:224] Handling call
0709 13:04:14.486890 (+  2318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.497620 19214 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:14.488895 19284 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:14.498607 18828 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:14.499285 18828 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:14.499840 18828 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.500021 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:14.500705 19285 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:14.501181 19285 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.501672 19285 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.501155 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:14.504020 19284 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:14.491641 19283 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.506263 19283 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.509227 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.510141 15158 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:14.509672 19285 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.511210 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.511804 15158 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.512414 19285 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:14.513935 15840 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:14.506688 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.514667 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1077) took 3912 ms. Trace:
I20260709 13:04:14.515643 15840 rpcz_store.cc:276] 0709 13:04:10.602605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.513404 (+3910799us) service_pool.cc:224] Handling call
0709 13:04:14.513749 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.514397 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.514607 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.512269 18828 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.521412 15159 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:14.530686 19284 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:14.532173 19284 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:14.541719 15848 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 3.888s	user 0.012s	sys 0.026s
I20260709 13:04:14.542346 15848 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:04:14.543104 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1069) took 4079 ms (client timeout 2147 ms). Trace:
W20260709 13:04:14.561609 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1080) took 3894 ms (client timeout 1931 ms). Trace:
W20260709 13:04:14.562234 15839 rpcz_store.cc:269] 0709 13:04:10.666860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.710669 (+ 43809us) service_pool.cc:224] Handling call
0709 13:04:14.561495 (+3850826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.565496 15158 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:14.565675 18828 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:14.543563 15848 rpcz_store.cc:269] 0709 13:04:10.463873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.595189 (+131316us) service_pool.cc:224] Handling call
0709 13:04:14.543006 (+3947817us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.566928 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 995) took 3894 ms. Trace:
I20260709 13:04:14.566810 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.567904 19281 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:14.568655 19281 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.569751 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.569824 19281 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.570653 15161 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:14.567413 15088 rpcz_store.cc:276] 0709 13:04:10.672102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.675531 (+  3429us) service_pool.cc:224] Handling call
0709 13:04:14.490493 (+3814962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.490890 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.490917 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.491442 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.561074 (+ 69632us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:14.561122 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.561207 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.566688 (+  5481us) raft_consensus.cc:1685] finished
0709 13:04:14.566707 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.566858 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":147,"thread_start_us":69184,"threads_started":1,"wal-append.queue_time_us":69729}
I20260709 13:04:14.571774 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.572410 15161 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.578155 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.579716 19280 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:14.580236 19280 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:14.580693 19280 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.581881 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.582536 15232 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:04:14.583681 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.584285 15232 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:14.589702 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.589979 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1079) took 3982 ms. Trace:
I20260709 13:04:14.591205 19078 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:11.013978 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 475106us
I20260709 13:04:14.591972 19078 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.592819 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:14.597384 18828 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:14.597745 15161 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:14.597965 18828 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:14.600492 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:14.601699 19288 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:14.602334 19288 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:14.603544 19288 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.567899 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.605336 15158 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:14.594362 15234 raft_consensus.cc:1217] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-9.8]   Dedup: 4.6->[7.7-9.8]
I20260709 13:04:14.605782 15234 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 6
I20260709 13:04:14.606097 15234 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 6.7
W20260709 13:04:14.591974 15052 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.626s (SENT) [suppressed 12 similar messages]
I20260709 13:04:14.594508 15840 rpcz_store.cc:276] 0709 13:04:10.607753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.516460 (+3908707us) service_pool.cc:224] Handling call
0709 13:04:14.516873 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.517586 (+   713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.517623 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.518146 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.588681 (+ 70535us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:14.588725 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.588807 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.589756 (+   949us) raft_consensus.cc:1685] finished
0709 13:04:14.589773 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.589917 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":70096,"threads_started":1,"wal-append.queue_time_us":61299}
I20260709 13:04:14.606385 15234 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.596422 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.613742 15232 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:14.606383 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.615420 15158 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:14.615258 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.631690 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1073) took 4091 ms. Trace:
I20260709 13:04:14.598852 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.613559 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1081) took 3821 ms. Trace:
I20260709 13:04:14.639901 19283 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.640561 19283 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.636783 15839 rpcz_store.cc:276] 0709 13:04:10.792189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.563044 (+3770855us) service_pool.cc:224] Handling call
0709 13:04:14.563791 (+   747us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:14.564420 (+   629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.564455 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:14.565632 (+  1177us) log.cc:844] Serialized 598 byte log entry
0709 13:04:14.613174 (+ 47542us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:14.613217 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.613305 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.613357 (+    52us) raft_consensus.cc:1685] finished
0709 13:04:14.613368 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.613500 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":47075,"threads_started":1,"wal-append.queue_time_us":44650}
I20260709 13:04:14.632500 15845 rpcz_store.cc:276] 0709 13:04:10.540629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.482413 (+3941784us) service_pool.cc:224] Handling call
0709 13:04:14.485290 (+  2877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.486171 (+   881us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:14.566860 (+ 80689us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.631200 (+ 64340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.631263 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:14.631317 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.631412 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.631601 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78492,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"thread_start_us":144230,"threads_started":2}
I20260709 13:04:14.646392 19299 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:14.648571 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:14.650236 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.593101 19284 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:14.651576 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.652730 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:14.652782 15232 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:14.594152 19280 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.647240 19299 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.656039 15228 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:14.656291 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.657392 15161 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.652271 19284 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.661567 19284 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:14.658295 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 66365.4us
I20260709 13:04:14.660446 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:14.663434 15052 connection.cc:256] RPC call timeout handler was delayed by -3.48865124200006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 13:04:14.663837 15052 connection.cc:256] RPC call timeout handler was delayed by -3.4436512419999872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:04:14.664155 15052 connection.cc:256] RPC call timeout handler was delayed by -3.4427512419999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:04:14.664463 15052 connection.cc:256] RPC call timeout handler was delayed by -3.4355512419999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
I20260709 13:04:14.658277 19280 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.664675 19278 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:14.655318 19299 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.673278 19278 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.664939 19301 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:14.669126 15161 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:14.669705 15158 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:14.665434 18827 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.666880 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:14.665500 15052 connection.cc:256] RPC call timeout handler was delayed by -3.2384512419999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 13:04:14.680127 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.680186 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.681488 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.682230 19298 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:14.682736 19298 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.683022 18827 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.683554 19298 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.684993 15231 raft_consensus.cc:2368] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:14.687947 15161 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:14.689378 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.690212 15846 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:04:14.690294 19278 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.690650 15846 pending_rounds.cc:85] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 8
I20260709 13:04:14.690378 15839 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:14.691926 15846 pending_rounds.cc:107] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:04:14.690791 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.692444 15846 raft_consensus.cc:2889] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.693375 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.694089 15161 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:14.694511 15848 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:14.694751 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.695385 15158 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:14.696698 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.691776 19278 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:14.699542 15840 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:14.699745 19278 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.698817 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:14.698859 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:14.702350 15231 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:14.677994 19301 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:14.689159 19281 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:14.707333 19281 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.708277 19281 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:14.707144 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.712554 15231 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:14.715962 15158 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:14.717525 18827 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.701103 19280 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:11.153574 15847 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:14.733548 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1072) took 4193 ms. Trace:
I20260709 13:04:14.719825 19278 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.719648 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:14.706009 19301 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:14.721539 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.726111 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.734011 15847 rpcz_store.cc:276] 0709 13:04:10.540310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.152764 (+612454us) service_pool.cc:224] Handling call
0709 13:04:11.153136 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.733195 (+3580059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.733475 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.736191 15235 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:14.737306 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:14.739426 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.740800 15158 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:14.748234 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:14.739140 19302 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:14.771807 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:14.773387 15157 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:14.774786 18827 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:14.774922 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.775228 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:14.775677 15157 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:14.775446 18827 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.776993 19302 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.777704 18827 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.777536 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:10.743288 15083 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:04:14.778012 19302 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.779067 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 998) took 4036 ms. Trace:
I20260709 13:04:14.779417 15083 rpcz_store.cc:276] 0709 13:04:10.742406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.742564 (+   158us) service_pool.cc:224] Handling call
0709 13:04:10.742973 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.778799 (+4035826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.779016 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:14.772589 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:14.784799 15052 connection.cc:256] RPC call timeout handler was delayed by -2.7262227200000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 13:04:14.785297 15052 connection.cc:256] RPC call timeout handler was delayed by -2.525808397999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:04:14.785591 15052 connection.cc:256] RPC call timeout handler was delayed by -2.4644512419999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3069s for a response
I20260709 13:04:14.784953 15231 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.752617 19284 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:14.865213 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.805305 15235 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:14.841763 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 64299.8us
W20260709 13:04:14.848417 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59842.8us
I20260709 13:04:14.773569 19278 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:14.789629 19298 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:14.862385 19280 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:14.872987 15161 raft_consensus.cc:2368] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:14.784507 19299 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:14.864804 19311 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:04:14.870608 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:14.873579 19280 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:14.883162 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.878189 19311 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:04:14.891260 18828 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.876987 19298 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.893275 19298 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.594421 19078 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.898240 15231 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:14.899767 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:14.899950 19301 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.901000 19288 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:14.901628 19288 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:14.902369 19301 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.902748 19288 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.913898 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.887171 19311 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:04:14.915607 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:14.916043 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.875103 19299 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:14.917081 15231 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:14.923236 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.924010 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.920825 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1059) took 4815 ms. Trace:
I20260709 13:04:14.924935 15841 rpcz_store.cc:276] 0709 13:04:10.105110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.105955 (+   845us) service_pool.cc:224] Handling call
0709 13:04:10.106332 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.669373 (+4563041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.920524 (+251151us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54006dc254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:14.920728 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380185344}
I20260709 13:04:14.918531 19311 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:14.926805 19311 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:14.927609 15231 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:14.928622 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.931591 19298 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:14.923285 19299 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.937798 19298 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:14.937578 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:14.933282 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:14.940696 15814 connection.cc:256] RPC call timeout handler was delayed by -3.4207379519999677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:04:14.941140 15814 connection.cc:256] RPC call timeout handler was delayed by -3.3685379519999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 13:04:14.941429 15814 connection.cc:256] RPC call timeout handler was delayed by -3.0598379520000663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 13:04:14.941743 15814 connection.cc:256] RPC call timeout handler was delayed by -2.8186379520000173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.242s for a response
I20260709 13:04:14.941848 15231 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:14.942075 15814 connection.cc:256] RPC call timeout handler was delayed by -2.6089379520000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26530000000000004s for a response
I20260709 13:04:14.942880 15235 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:14.877451 19278 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.946746 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:14.947536 15847 raft_consensus.cc:2368] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:14.947886 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.949391 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:14.955869 15848 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:14.955945 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:14.948662 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.948267 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.956008 15158 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.958316 15840 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:14.949730 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.917649 19310 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:14.947149 19311 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.961088 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.960685 19310 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.962162 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.962630 15848 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:14.963048 15845 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:14.963869 19311 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:14.968382 15051 connection.cc:256] RPC call timeout handler was delayed by -3.7753517680000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 13:04:14.974902 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:14.975106 15051 connection.cc:256] RPC call timeout handler was delayed by -3.7528517680000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:04:14.975642 15051 connection.cc:256] RPC call timeout handler was delayed by -3.72945176799999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:04:14.976046 15051 connection.cc:256] RPC call timeout handler was delayed by -3.6543390130000262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2663s for a response
W20260709 13:04:14.976722 15051 connection.cc:256] RPC call timeout handler was delayed by -3.1552241949999598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
W20260709 13:04:14.977106 15051 connection.cc:256] RPC call timeout handler was delayed by -2.3376995830000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:04:14.977389 15051 connection.cc:256] RPC call timeout handler was delayed by -1.8934517679999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40220000000000006s for a response
I20260709 13:04:14.943552 19285 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:14.964644 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.962913 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:14.976502 19310 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.981983 15158 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:14.984638 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:14.988054 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:14.989214 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:14.989652 19284 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:14.989917 15086 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.990272 19285 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:14.991209 19285 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:14.974395 19168 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:14.992959 15052 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff3342a5b2ad49e2b252d7cd47b9851d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:14.993194 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:14.993993 19278 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:14.994565 19278 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:14.996754 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:14.998390 15231 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:14.999362 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:14.999650 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:15.001812 15086 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:15.002630 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:15.003217 15086 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.007655 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:14.979418 19078 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.010120 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:15.010248 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:15.010248 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:15.011741 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.019578 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:15.020200 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:15.020949 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:15.021440 15848 raft_consensus.cc:2376] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7. Re-sending same reply.
I20260709 13:04:15.021790 15231 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:15.022589 15201 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 4.354s	user 0.033s	sys 0.024s
I20260709 13:04:15.003821 19299 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:15.024628 15201 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 4.370s	user 0.034s	sys 0.025s
W20260709 13:04:15.025002 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 4.40652e+06us
W20260709 13:04:15.029371 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.011270 19078 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:10.741709 15087 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:04:15.033370 15237 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.033849 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 996) took 4358 ms. Trace:
I20260709 13:04:15.034648 18827 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.035161 18827 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:15.034212 15087 rpcz_store.cc:276] 0709 13:04:10.675952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.676116 (+   164us) service_pool.cc:224] Handling call
0709 13:04:10.741390 (+ 65274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.033527 (+4292137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.033757 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:15.041730 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.037899 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:15.052078 15201 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.658s (SENT)
I20260709 13:04:15.052704 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:15.053663 19280 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:15.041754 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:15.054248 15154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 818) took 4499 ms (client timeout 2653 ms). Trace:
I20260709 13:04:15.054781 15157 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:15.054646 15154 rpcz_store.cc:269] 0709 13:04:10.543718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.553875 (+ 10157us) service_pool.cc:224] Handling call
0709 13:04:15.043163 (+4489288us) spinlock_profiling.cc:243] Waited 3.86 s on lock 0x7b540013e2d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:15.043343 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8880693504}
I20260709 13:04:15.057641 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:15.065723 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:15.070871 15848 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:04:15.073648 19317 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.093s	user 0.001s	sys 0.002s
I20260709 13:04:15.013240 14974 maintenance_manager.cc:643] P df53447e208f455db54704b341851d53: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 15.397s	user 0.219s	sys 0.142s Metrics: {"bytes_written":29423,"cfile_init":1,"dirs.queue_time_us":1330127,"dirs.run_cpu_time_us":477,"dirs.run_wall_time_us":1964,"drs_written":1,"lbm_read_time_us":24300,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":2,"lbm_write_time_us":384627,"lbm_writes_10-100_ms":12,"lbm_writes_lt_1ms":18,"peak_mem_usage":0,"rows_written":63,"spinlock_wait_cycles":2461952,"thread_start_us":1365918,"threads_started":2,"wal-append.queue_time_us":635084,"wal-append.run_cpu_time_us":801,"wal-append.run_wall_time_us":1138390}
I20260709 13:04:15.032675 19078 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:15.074404 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:15.075038 15157 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:15.076855 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:15.078058 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:15.078956 15231 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 7: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.080112 19283 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:15.034531 19299 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:15.081168 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.082966 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:15.083186 15236 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:15.083304 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.042037 19284 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.083896 15237 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:15.086270 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:15.090668 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 95442.8us
I20260709 13:04:15.087766 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:15.091658 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.091933 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:15.098906 15814 leader_election.cc:336] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.809s (SENT)
I20260709 13:04:15.091838 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:15.099313 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:15.099745 15236 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.100406 15231 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.100541 15087 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:15.093066 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:15.088121 19284 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:15.101302 15079 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:15.102257 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:15.101367 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:15.105664 15231 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.106108 14975 maintenance_manager.cc:419] P df53447e208f455db54704b341851d53: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10393 bytes on disk
I20260709 13:04:15.108526 14974 maintenance_manager.cc:643] P df53447e208f455db54704b341851d53: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":341,"lbm_reads_lt_1ms":4}
I20260709 13:04:15.111244 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:15.115119 19284 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:15.137428 15201 connection.cc:256] RPC call timeout handler was delayed by -1.1059257040000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3496s for a response
I20260709 13:04:15.141520 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:15.144136 19310 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:15.144757 19310 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.145313 19310 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:15.147783 15237 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:15.150848 15236 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:15.119377 19319 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 13:04:15.154676 15201 leader_election.cc:336] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.413s (SENT)
I20260709 13:04:15.163941 15237 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:15.164635 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:15.165723 15237 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:15.166183 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:15.167120 15162 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:15.167447 15237 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:15.168928 15237 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:15.169206 19307 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:15.169296 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:15.169673 19307 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:15.170015 15237 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:15.170560 19307 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:15.171320 15237 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:04:15.171829 19307 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:15.172498 15237 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:15.174034 15162 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:15.181805 15231 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:15.157505 19310 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:15.151939 19322 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:04:15.183257 19280 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.173063 19299 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:15.191632 19280 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:15.175143 19298 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.190215 15231 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:15.191100 19310 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:15.192054 15234 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:15.152521 19278 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:04:15.192832 19298 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:15.196869 19311 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:15.198971 19298 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:15.209898 15237 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:04:15.212178 15201 connection.cc:256] RPC call timeout handler was delayed by -4.5712768409998716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260709 13:04:15.213939 19310 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:15.214399 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 56382.1us
W20260709 13:04:15.212625 15201 connection.cc:256] RPC call timeout handler was delayed by -4.4326768409998749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:04:15.215740 15201 connection.cc:256] RPC call timeout handler was delayed by -4.3669768409998824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
W20260709 13:04:15.216185 15201 connection.cc:256] RPC call timeout handler was delayed by -4.3408768409999539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:04:15.217291 15231 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:15.307343 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.308555 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.309058 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.309491 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.474143 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.476573 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.631644 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08001fdf80 after lost signal to thread 13970
W20260709 13:04:15.907567 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.982897 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.511091 15230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.298s	user 0.008s	sys 0.009s
W20260709 13:04:16.511689 15230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.009s	sys 0.009s
W20260709 13:04:16.446043 19311 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.249s	user 0.003s	sys 0.005s
I20260709 13:04:16.497911 19283 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.608209 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.627133 15234 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.431s	user 0.002s	sys 0.009s
I20260709 13:04:16.606385 19311 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:16.630091 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.41079e+06us
W20260709 13:04:16.630632 15201 connection.cc:256] RPC call timeout handler was delayed by -4.2472768409999162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260709 13:04:16.631088 15201 connection.cc:256] RPC call timeout handler was delayed by -4.134776840999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:04:16.631269 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 397540us
W20260709 13:04:16.631551 15201 connection.cc:256] RPC call timeout handler was delayed by -4.08437684099988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20450000000000002s for a response
W20260709 13:04:16.632053 15201 connection.cc:256] RPC call timeout handler was delayed by -4.0798768409998729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 13:04:16.625432 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.439s	user 0.000s	sys 0.047s
W20260709 13:04:16.632443 15201 connection.cc:256] RPC call timeout handler was delayed by -4.01417684099988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 13:04:16.632582 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.642601 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.457s	user 0.000s	sys 0.047s
W20260709 13:04:16.642601 15201 connection.cc:256] RPC call timeout handler was delayed by -3.9925768409998454s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
I20260709 13:04:16.633270 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1912) took 2181 ms. Trace:
W20260709 13:04:16.644268 15201 connection.cc:256] RPC call timeout handler was delayed by -3.9700768409999228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260709 13:04:16.644981 15230 rpcz_store.cc:276] 0709 13:04:14.451603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.210747 (+759144us) service_pool.cc:224] Handling call
0709 13:04:15.212020 (+  1273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.212263 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:15.212553 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.632832 (+1420279us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:16.632910 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:16.632981 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.633064 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.633208 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"thread_start_us":1419803,"threads_started":2,"wal-append.queue_time_us":1432591}
I20260709 13:04:16.648358 19311 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.665346 19311 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:16.645037 15201 connection.cc:256] RPC call timeout handler was delayed by -3.9430768409998791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260709 13:04:16.667235 15201 connection.cc:256] RPC call timeout handler was delayed by -3.9331768409998631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:04:16.667737 15201 connection.cc:256] RPC call timeout handler was delayed by -3.2712258819999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:04:16.668128 15201 connection.cc:256] RPC call timeout handler was delayed by -3.1227258819999406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
W20260709 13:04:16.668524 15201 connection.cc:256] RPC call timeout handler was delayed by -3.1146258820000412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 13:04:16.668888 15201 connection.cc:256] RPC call timeout handler was delayed by -2.8725258819999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 13:04:16.669251 15201 connection.cc:256] RPC call timeout handler was delayed by -2.7402258820000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:04:16.669486 19337 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:16.670433 19337 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:16.685065 19307 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.513s	user 0.009s	sys 0.003s
W20260709 13:04:16.669610 15201 connection.cc:256] RPC call timeout handler was delayed by -2.0580258820000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2856s for a response
W20260709 13:04:16.691726 15079 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.590s	user 0.012s	sys 0.017s
I20260709 13:04:16.692250 15079 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:16.668143 19311 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:16.625988 19283 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:16.695209 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 1952) took 1600 ms (client timeout 2078 ms). Trace:
W20260709 13:04:16.697903 15079 rpcz_store.cc:269] 0709 13:04:15.092451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.092818 (+   367us) service_pool.cc:224] Handling call
0709 13:04:16.692871 (+1600053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.703032 19337 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:16.713866 19206 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.714702 19206 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.715826 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 828) took 1613 ms. Trace:
I20260709 13:04:16.716240 15162 rpcz_store.cc:276] 0709 13:04:15.097018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.174628 (+ 77610us) service_pool.cc:224] Handling call
0709 13:04:15.185450 (+ 10822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.195094 (+  9644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.195123 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.209165 (+ 14042us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.695947 (+1486782us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:16.695991 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.696068 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.709928 (+ 13860us) raft_consensus.cc:1685] finished
0709 13:04:16.709946 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.710092 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":421,"raft.queue_time_us":1206,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":184604,"threads_started":1,"wal-append.queue_time_us":1489037}
I20260709 13:04:16.717942 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1914) took 2265 ms. Trace:
I20260709 13:04:16.718729 19283 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:10.811542 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.718465 15228 rpcz_store.cc:276] 0709 13:04:14.452176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.627469 (+175293us) service_pool.cc:224] Handling call
0709 13:04:14.627948 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.656462 (+ 28514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.717600 (+2061138us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5400175a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:16.717861 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64122112}
I20260709 13:04:16.731132 19206 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:16.737768 19206 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.738601 19206 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.741386 19206 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:16.744026 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2140) took 2254 ms. Trace:
I20260709 13:04:16.744472 15234 rpcz_store.cc:276] 0709 13:04:14.490126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.191372 (+701246us) service_pool.cc:224] Handling call
0709 13:04:15.191725 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.191937 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:16.627748 (+1435811us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.743504 (+115756us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.743700 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:16.743746 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:16.743791 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.743865 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.743974 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18084,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":106112,"threads_started":2,"wal-append.queue_time_us":73176}
I20260709 13:04:16.755303 19078 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:16.766819 15054 connection.cc:256] RPC call timeout handler was delayed by -5.637435374000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260709 13:04:16.768673 19307 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.769254 19307 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.771462 19307 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:16.772073 15054 connection.cc:256] RPC call timeout handler was delayed by -4.9621822849999262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 13:04:16.772894 15054 connection.cc:256] RPC call timeout handler was delayed by -4.797721878999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 13:04:16.773527 15054 connection.cc:256] RPC call timeout handler was delayed by -4.635735373999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3069s for a response
W20260709 13:04:16.773895 15054 connection.cc:256] RPC call timeout handler was delayed by -4.634412609999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:04:16.774262 15054 connection.cc:256] RPC call timeout handler was delayed by -4.5957622360000414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:04:16.774626 15054 connection.cc:256] RPC call timeout handler was delayed by -4.3338126099999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 13:04:16.774982 15054 connection.cc:256] RPC call timeout handler was delayed by -3.7780353739999555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40220000000000006s for a response
I20260709 13:04:16.776688 15081 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:16.777863 19078 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:16.778853 19078 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.782136 19078 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:16.784775 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 829) took 1675 ms. Trace:
I20260709 13:04:16.785285 15162 rpcz_store.cc:276] 0709 13:04:15.109139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.721119 (+1611980us) service_pool.cc:224] Handling call
0709 13:04:16.721478 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.721854 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.721884 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.722337 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.743947 (+ 21610us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:16.783972 (+ 40025us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.784454 (+   482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.784534 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.784572 (+    38us) raft_consensus.cc:1685] finished
0709 13:04:16.784590 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.784722 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8113,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6578,"thread_start_us":58222,"threads_started":2,"wal-append.queue_time_us":22900}
W20260709 13:04:16.796594 15157 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.717s	user 0.007s	sys 0.022s
W20260709 13:04:16.799319 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.698s	user 0.016s	sys 0.028s
I20260709 13:04:16.797204 15157 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:04:16.807245 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.707s	user 0.018s	sys 0.029s
I20260709 13:04:16.806834 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 824) took 1752 ms. Trace:
W20260709 13:04:16.808023 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.70769e+06us
I20260709 13:04:16.807991 15157 rpcz_store.cc:276] 0709 13:04:15.054449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.074140 (+ 19691us) service_pool.cc:224] Handling call
0709 13:04:16.806713 (+1732573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.808928 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:16.809815 19078 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:16.809967 15051 leader_election.cc:400] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:16.810534 15051 leader_election.cc:403] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.811357 19307 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:16.818262 15843 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:04:16.820127 15843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1078) took 6213 ms (client timeout 1982 ms). Trace:
W20260709 13:04:10.957474 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 423431us
I20260709 13:04:16.822697 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:16.823295 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:16.823788 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:16.824590 19206 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:16.825395 19206 consensus_queue.cc:1243] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:16.826815 19206 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:16.827052 19252 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:16.827342 19206 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:16.827592 19252 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:16.828325 19206 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.829034 19281 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:16.829602 19281 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:16.820463 15843 rpcz_store.cc:269] 0709 13:04:10.607252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.649078 (+ 41826us) service_pool.cc:224] Handling call
0709 13:04:16.819908 (+6170830us) spinlock_profiling.cc:243] Waited 5.67 s on lock 0x7b5400730354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:16.820037 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13032911616}
I20260709 13:04:16.830477 19281 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:16.829051 19340 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:16.832322 19340 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:16.833493 19340 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.835083 15156 raft_consensus.cc:2410] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:04:16.835641 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.835683 19337 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.835973 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 814) took 6380 ms (client timeout 1809 ms). Trace:
W20260709 13:04:16.836372 15156 rpcz_store.cc:269] 0709 13:04:10.455446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.461001 (+  5555us) service_pool.cc:224] Handling call
0709 13:04:16.835761 (+6374760us) spinlock_profiling.cc:243] Waited 6.37 s on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:16.835912 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14656641408}
I20260709 13:04:16.836419 19337 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:16.838624 19337 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:16.828445 19252 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.840126 19252 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:16.840577 19307 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:16.843444 19078 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.843868 19078 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:16.847455 15201 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.585s (SENT)
W20260709 13:04:16.848552 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.848552 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:16.850137 19307 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.849470 15201 leader_election.cc:336] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.604s (SENT)
I20260709 13:04:16.850669 19307 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:16.851379 19347 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:16.852820 19307 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:16.852108 19347 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:16.854327 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:16.854945 15079 raft_consensus.cc:2376] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10. Re-sending same reply.
I20260709 13:04:16.854524 19347 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:16.855059 15841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1076) took 6037 ms (client timeout 1989 ms). Trace:
I20260709 13:04:16.855623 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.855937 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:16.855460 15841 rpcz_store.cc:269] 0709 13:04:10.817676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.854873 (+6037197us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:16.854978 (+   105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:16.856964 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.857044 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:16.857393 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.857520 15840 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:16.858072 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:16.858361 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.858510 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.859061 15840 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.859210 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:16.859579 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.860108 15841 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:16.860157 15840 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:04:16.860569 15840 pending_rounds.cc:85] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 7
I20260709 13:04:11.153129 15844 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:04:16.860819 15840 pending_rounds.cc:107] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:04:16.880153 15840 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.886514 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:16.888723 15201 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.689s (SENT)
I20260709 13:04:16.861167 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.890630 15840 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.873675 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:16.894521 15054 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.626s (SENT)
I20260709 13:04:16.874624 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1039) took 6359 ms. Trace:
I20260709 13:04:16.876205 15155 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:16.863065 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:16.890823 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.897193 15088 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 6.
I20260709 13:04:16.861585 19214 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.903808 19214 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:16.905234 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.906105 19214 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:16.861666 15079 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:16.861866 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:16.907877 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46085.8us
I20260709 13:04:16.862028 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:16.862751 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:16.867799 15841 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:16.895841 15844 rpcz_store.cc:276] 0709 13:04:10.515556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.152263 (+636707us) service_pool.cc:224] Handling call
0709 13:04:11.152626 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.874253 (+5721627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.874551 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.911461 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.863464 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:16.894932 15843 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:16.863251 19078 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:16.914077 15054 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.895s (SENT)
I20260709 13:04:16.913426 15844 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:16.918533 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.919140 15081 raft_consensus.cc:2410] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:16.919034 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.919631 15848 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:16.924461 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:16.919883 15844 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.926106 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.927367 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.926608 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.927630 15848 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.928079 15840 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:16.928779 15846 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:16.928926 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.929562 15843 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.930786 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.925170 19214 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:16.931725 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.932361 15848 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:16.933187 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.934114 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.934768 15841 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:16.935105 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.935634 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.936074 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.936668 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.938865 15841 raft_consensus.cc:2376] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:04:16.931771 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.938220 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.941197 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.941618 19353 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:16.942091 19353 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.942440 19353 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:16.942660 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.945542 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.946768 19078 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:16.947616 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.948516 15841 raft_consensus.cc:2376] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:04:16.949709 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.950693 15841 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:16.951773 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:10.749818 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:16.952543 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:16.953370 15054 leader_election.cc:336] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 2.229s (SENT)
I20260709 13:04:16.953624 15848 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.954852 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.958689 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.959278 15846 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:16.959730 15844 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.960762 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.957254 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:16.962648 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.963725 19078 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:16.963929 15846 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:16.961912 15844 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:16.965838 15846 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:16.966578 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.967919 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.968536 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.968699 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:16.970026 15840 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:16.968452 15843 raft_consensus.cc:2376] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:04:16.970871 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.972074 15845 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:16.976418 15082 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:16.977734 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:16.979787 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:16.972951 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:16.981802 15082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 997) took 6300 ms (client timeout 1625 ms). Trace:
I20260709 13:04:16.981701 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:16.982378 15082 rpcz_store.cc:269] 0709 13:04:10.676319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.749528 (+ 73209us) service_pool.cc:224] Handling call
0709 13:04:16.976919 (+6227391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.984349 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.986444 15843 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:16.987470 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:16.988164 15154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 826) took 1906 ms (client timeout 2011 ms). Trace:
W20260709 13:04:16.988525 15154 rpcz_store.cc:269] 0709 13:04:15.081357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.105294 (+ 23937us) service_pool.cc:224] Handling call
0709 13:04:16.988110 (+1882816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.989470 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:16.990579 15845 raft_consensus.cc:2376] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7. Re-sending same reply.
I20260709 13:04:16.982710 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:16.993134 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:16.993230 19206 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:16.993844 15840 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:04:16.994630 19332 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.341s	user 0.002s	sys 0.000s
I20260709 13:04:16.988783 19353 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:16.997118 19353 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:17.000810 15052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.908s	user 0.014s	sys 0.033s
W20260709 13:04:17.001458 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.909s	user 0.015s	sys 0.033s
W20260709 13:04:17.001837 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.90891e+06us
I20260709 13:04:17.003924 19078 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:17.005863 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 81523.8us
I20260709 13:04:17.007229 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.008002 19347 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:17.008375 19307 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:04:17.010573 19353 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:17.012143 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.012730 15082 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:17.009459 19214 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:17.015733 19214 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.017757 15052 leader_election.cc:400] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:17.018121 15052 leader_election.cc:403] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:04:17.030972 15082 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:17.038337 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:17.038906 15813 connection.cc:256] RPC call timeout handler was delayed by -4.9070785990001013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 13:04:17.039503 19353 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:17.040056 19353 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:17.040968 19353 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:17.042851 15155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 811) took 6588 ms (client timeout 2420 ms). Trace:
W20260709 13:04:17.043270 15155 rpcz_store.cc:269] 0709 13:04:10.454630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.494675 (+ 40045us) service_pool.cc:224] Handling call
0709 13:04:17.042658 (+6547983us) spinlock_profiling.cc:243] Waited 6.32 s on lock 0x7b54000857d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:17.042786 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14526392576}
I20260709 13:04:17.047076 19255 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.047750 19255 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.055393 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:17.057216 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.057940 15088 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.059437 19357 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:17.059897 19357 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:17.060271 19357 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:17.060936 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.061174 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.061761 15080 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:17.062858 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.056660 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.064770 15079 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:17.066233 15081 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.068622 19357 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.071097 19357 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:17.071736 19357 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.074545 15081 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 9. Ops: []
I20260709 13:04:17.075760 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.077929 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.083988 15088 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:17.084288 15079 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:17.085410 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.088984 15082 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.072316 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.083550 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.078145 19255 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.086189 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.092245 19237 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.095194 15080 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:17.088267 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.095254 19237 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.096169 15079 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.097749 19237 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:17.098137 19255 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:17.098835 19237 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:17.099743 19237 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:17.100054 19237 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.100384 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:17.101258 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.102406 19255 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:17.103210 19255 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.105816 19255 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:17.107366 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 6.063s	user 0.015s	sys 0.021s
W20260709 13:04:17.107787 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 6.064s	user 0.016s	sys 0.022s
I20260709 13:04:17.100989 19237 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.109759 19237 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:17.114205 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53347.8us
I20260709 13:04:17.115038 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.117998 19237 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:17.118300 19255 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:17.118435 19237 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.118832 19255 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.118963 19237 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.119792 19255 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.121740 15158 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.122747 19363 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:17.123497 19363 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.124164 19206 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.124545 19363 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.126979 19307 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:17.126657 19237 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.129264 19237 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:17.130436 19333 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.476s	user 0.001s	sys 0.002s
I20260709 13:04:17.134253 19255 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:17.137169 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.139072 19357 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:17.139667 19357 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.146621 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.148866 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:17.150993 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.153198 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.158932 15155 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.161231 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.162180 15155 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:17.162082 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2688) took 2647 ms (client timeout 2309 ms). Trace:
W20260709 13:04:17.163596 15161 rpcz_store.cc:269] 0709 13:04:14.514737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.696497 (+181760us) service_pool.cc:224] Handling call
0709 13:04:17.161878 (+2465381us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:17.162012 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403598720}
I20260709 13:04:17.164577 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.168545 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.169468 15085 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:17.175161 15158 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:17.176505 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.179780 15155 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:17.185722 15158 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.187480 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1935) took 2203 ms. Trace:
I20260709 13:04:17.188046 15083 rpcz_store.cc:276] 0709 13:04:14.983920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.984078 (+   158us) service_pool.cc:224] Handling call
0709 13:04:15.004289 (+ 20211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.081380 (+ 77091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.081412 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.087412 (+  6000us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.187054 (+2099642us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:17.187109 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.187198 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.187255 (+    57us) raft_consensus.cc:1685] finished
0709 13:04:17.187273 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.187408 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94789,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":17019,"thread_start_us":180646,"threads_started":2,"wal-append.queue_time_us":2001781}
I20260709 13:04:17.191071 15159 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.197109 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.204051 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.205238 15161 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9.
I20260709 13:04:17.208019 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2128) took 2728 ms. Trace:
I20260709 13:04:17.208483 15236 rpcz_store.cc:276] 0709 13:04:14.479266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.158432 (+679166us) service_pool.cc:224] Handling call
0709 13:04:15.159151 (+   719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.159625 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.159668 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.175344 (+ 15676us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.644323 (+1468979us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:16.644361 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.644429 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.112443 (+468014us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:17.207705 (+ 95262us) raft_consensus.cc:1685] finished
0709 13:04:17.207729 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.207961 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":1154,"spinlock_wait_cycles":13753088,"thread_start_us":1459119,"threads_started":1,"wal-append.queue_time_us":1465399}
W20260709 13:04:17.195695 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1075) took 6516 ms (client timeout 1731 ms). Trace:
W20260709 13:04:17.215695 15842 rpcz_store.cc:269] 0709 13:04:10.679755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.729890 (+ 50135us) service_pool.cc:224] Handling call
0709 13:04:17.195605 (+6465715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:17.221208 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.208073 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.232923 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.239449 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.240635 15085 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.243966 15161 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 5. Current term is 7. Ops: []
I20260709 13:04:17.243731 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.246517 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.248943 15079 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:17.250499 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.253891 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.261752 19357 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:17.262917 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.263571 15158 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:17.267832 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08003191c0 after lost signal to thread 15020
W20260709 13:04:17.287461 15869 debug-util.cc:398] Leaking SignalData structure 0x7b0800281a00 after lost signal to thread 15034
I20260709 13:04:17.278831 15162 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:04:17.279420 15157 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:04:17.289896 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.280015 15156 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:04:17.282886 15160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2675) took 2775 ms (client timeout 2223 ms). Trace:
W20260709 13:04:17.292227 15160 rpcz_store.cc:269] 0709 13:04:14.507508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.282612 (+2775104us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:17.282788 (+   176us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:17.286923 15158 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:17.294885 15161 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:17.296888 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.297689 15161 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:17.298729 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.300370 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.318354 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.301177 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.319536 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.319736 15156 raft_consensus.cc:2376] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10. Re-sending same reply.
I20260709 13:04:17.320724 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.302265 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.322299 15157 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:17.275794 19253 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.323764 15161 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.302830 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.264794 19280 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:17.308957 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.312330 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.316174 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.320794 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:17.327184 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 225992us
I20260709 13:04:17.327649 15154 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.327845 15156 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:17.329042 15127 connection.cc:256] RPC call timeout handler was delayed by -5.4212865189999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:04:17.329445 15127 connection.cc:256] RPC call timeout handler was delayed by -5.3798865190000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:04:17.329764 15127 connection.cc:256] RPC call timeout handler was delayed by -5.328586519000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 13:04:17.330071 15127 connection.cc:256] RPC call timeout handler was delayed by -5.1710865190000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:04:17.330363 15127 connection.cc:256] RPC call timeout handler was delayed by -5.010886518999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
I20260709 13:04:17.330334 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:17.330929 15127 connection.cc:256] RPC call timeout handler was delayed by -4.9046865190000517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 13:04:17.331390 19205 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:17.332265 19255 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:17.332535 19205 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.332751 19255 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.333001 19205 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.333770 19255 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.333853 19205 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.335335 19205 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:17.336351 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.337625 15839 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.339466 19357 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:17.341073 19370 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:17.342167 19371 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:17.342952 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:17.343768 15163 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2673) took 2836 ms (client timeout 2856 ms). Trace:
I20260709 13:04:17.343818 19078 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:17.344177 15163 rpcz_store.cc:269] 0709 13:04:14.506997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.342664 (+2835667us) service_pool.cc:224] Handling call
0709 13:04:17.343705 (+  1041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:17.345449 15839 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:17.346035 19311 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.346673 19311 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.305280 15159 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:17.348181 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4280 after lost signal to thread 15112
W20260709 13:04:17.349505 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08001d60c0 after lost signal to thread 15187
I20260709 13:04:17.350234 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.352993 19253 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:17.353622 19253 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.354094 19253 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:17.354429 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.356623 15154 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:17.356909 15236 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 9. Ops: []
I20260709 13:04:17.359100 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:17.360075 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.361101 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.362142 15229 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:17.362287 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.362810 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1917) took 2909 ms. Trace:
I20260709 13:04:17.363150 15229 rpcz_store.cc:276] 0709 13:04:14.453549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.361653 (+2908104us) service_pool.cc:224] Handling call
0709 13:04:17.361964 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.362585 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.362759 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:17.364658 19363 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:17.369874 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.370577 15234 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:17.373837 15229 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:04:17.331296 15127 connection.cc:256] RPC call timeout handler was delayed by -4.8353865190000533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 13:04:17.399367 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.429162 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79017.4us
I20260709 13:04:17.431677 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:17.430547 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51891.5us
W20260709 13:04:17.429294 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08003cb620 after lost signal to thread 15261
W20260709 13:04:17.435233 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08003de0e0 after lost signal to thread 15262
I20260709 13:04:17.436796 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.439069 19356 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.439566 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:17.440100 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100046us
W20260709 13:04:17.443640 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.443650 15229 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:17.445323 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.445928 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:17.470768 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.472936 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.473635 15234 raft_consensus.cc:2410] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:04:17.476286 15201 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.992s (SENT)
I20260709 13:04:17.473600 15229 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.440970 19206 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:17.443507 19252 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:17.482759 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.484467 15234 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:17.488756 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 46615.9us
I20260709 13:04:17.492683 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.495393 19371 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.496014 19371 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.479466 19356 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.501586 15232 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.503283 15237 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:04:17.501792 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.502214 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.501650 15234 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.504948 15235 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.505573 15236 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:17.480461 19376 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.485427 19252 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.511106 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.525321 15229 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.511570 15087 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:17.512432 15230 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:17.519587 15234 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:04:17.521490 15232 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:17.485972 19380 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.528829 15234 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:17.535991 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54953.7us
I20260709 13:04:17.540017 15235 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:04:17.541565 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.542400 15232 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.545581 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.545783 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.546597 15229 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:17.509006 19355 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.554558 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.554828 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.555549 15234 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:17.526173 19252 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.555652 15230 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.509959 19376 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.569664 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.570667 15845 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:17.568677 15229 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.571851 15842 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:04:17.572245 15842 pending_rounds.cc:85] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 6
I20260709 13:04:17.572476 15842 pending_rounds.cc:107] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:04:17.572796 15842 raft_consensus.cc:2889] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:17.571408 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81894.4us
I20260709 13:04:17.529452 19380 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.573560 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:17.574136 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.576367 15229 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:17.577200 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.578228 15234 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:17.580543 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:17.582929 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.581188 15234 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:17.561395 19355 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.578409 19379 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:04:17.586136 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.586736 19375 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:17.590708 15085 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:17.593259 19376 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.594538 19376 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.595798 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:17.596637 19375 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:17.597174 19375 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.597476 19371 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:17.597653 19375 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.598425 19377 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:17.599526 19357 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:17.599920 19357 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.600790 19357 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.601099 19380 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.602125 19380 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.602227 19357 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.572429 19078 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:04:17.605598 19371 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:17.605983 19371 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.606681 15236 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.606741 19371 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.610603 19371 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:17.612255 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.614099 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.621973 19337 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.622325 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.589237 19370 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:17.623307 19386 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.625363 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.625365 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.626230 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.624357 19384 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:17.629719 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:17.623673 19370 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.576910 15087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "301fa97ba426402092eaf85685303b33"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:17.636538 19206 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.637017 15087 raft_consensus.cc:606] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.638159 19280 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:17.641346 19205 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:17.642160 18891 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:17.642553 18891 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.643352 18891 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.644564 18891 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:17.617486 19253 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.647023 19355 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.649339 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:17.649785 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.650460 15231 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:17.636581 19311 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.650995 15230 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:17.654008 15232 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:17.654246 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.648830 19253 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:17.651294 19379 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:17.656785 19379 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.656808 19280 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:17.657537 19280 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:17.657800 19379 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.658385 19280 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:17.660435 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.663273 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.665110 15229 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:17.668435 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.651412 19370 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.666978 19252 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.673089 19311 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:17.682382 19252 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.668282 19340 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.689258 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:17.704241 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:17.672497 19385 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:04:17.673569 19337 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:17.726173 19395 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:17.733585 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 108284us
I20260709 13:04:17.734416 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.724862 19252 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:17.738572 15085 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:17.741221 19386 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:17.741392 19205 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:17.742731 15087 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:04:17.742873 19366 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:17.743353 19366 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.745324 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.746650 19357 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:17.747659 19357 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:17.748288 19357 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:17.749446 19384 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:17.749468 19386 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.749892 19384 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.750077 19386 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.751214 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:17.750857 19384 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.751874 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.752293 19384 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.734020 19385 consensus_queue.cc:1243] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:17.727095 19253 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:17.757138 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.757282 15846 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.754230 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.758448 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.759294 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.756141 19253 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.760809 15841 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.761118 15079 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:17.757005 19206 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:04:17.762229 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.762902 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.763199 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:17.764420 15869 debug-util.cc:398] Leaking SignalData structure 0x7b080040d4e0 after lost signal to thread 15872
I20260709 13:04:17.766407 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.767520 15844 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:17.760607 19253 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.770581 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:17.738232 18827 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:17.780391 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.771921 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:17.772111 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.739450 19375 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.764531 19357 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:17.739754 19252 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:17.783916 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.783582 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.744247 19366 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.789088 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.786367 15079 raft_consensus.cc:2368] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:17.790337 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.788767 15844 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:17.784803 15840 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:17.794288 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.794786 15079 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.794842 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.785094 19395 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:04:17.797003 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.796732 19237 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:17.797610 15841 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.797842 19237 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.799296 19237 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.802642 19395 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:17.803287 19395 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:17.804235 19395 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.791664 19337 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.791518 19252 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:17.826326 15841 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:17.817387 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:17.833323 15079 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:17.834201 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.836045 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 167644us
I20260709 13:04:17.837829 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.837739 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:17.787899 19206 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:17.750883 19386 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.847545 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.849478 15079 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:17.841378 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.831645 19255 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:17.851770 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.841132 19206 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.854496 15126 connection.cc:256] RPC call timeout handler was delayed by -1.6093422599999485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 13:04:17.836170 19375 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:17.854945 15126 connection.cc:256] RPC call timeout handler was delayed by -1.5634422599998743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:04:17.875264 15126 connection.cc:256] RPC call timeout handler was delayed by -1.5058490989999882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:04:17.875726 15126 connection.cc:256] RPC call timeout handler was delayed by -1.4833490989999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 13:04:17.876101 15126 connection.cc:256] RPC call timeout handler was delayed by -1.4432766499999161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 13:04:17.876499 15126 connection.cc:256] RPC call timeout handler was delayed by -1.4311422599998878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:04:17.836606 19357 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.839166 19366 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:17.857918 15840 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:17.858364 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.876830 15126 connection.cc:256] RPC call timeout handler was delayed by -1.4104422599998543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 13:04:17.845544 19337 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.897365 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:17.897738 15126 connection.cc:256] RPC call timeout handler was delayed by -1.2394490990000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 13:04:17.898068 15126 connection.cc:256] RPC call timeout handler was delayed by -1.094491441999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
I20260709 13:04:17.899524 15079 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:17.900558 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:17.901593 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.902123 15079 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:17.903007 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.904065 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.904628 15079 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:17.884948 19206 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.915316 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:17.915992 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78246.1us
I20260709 13:04:17.917886 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:17.919127 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.921085 15079 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:17.915292 19255 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:17.921793 19255 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.922219 19255 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:17.922981 19397 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:17.923589 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:17.924647 19357 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.926789 15154 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.929028 15163 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:04:17.932204 19386 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.938537 19211 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:17.939148 19211 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.939482 15846 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:17.940094 19211 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:17.952664 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.895058 19253 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.955087 19253 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.936630 19252 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.962716 19252 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:17.964054 19307 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.964505 19307 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.965392 19307 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.966737 19307 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.967269 15844 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:17.968334 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.968686 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:17.969203 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.970518 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.971422 19355 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:17.972170 19355 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.973210 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:17.973909 15840 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:17.973760 19387 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:17.974695 19387 consensus_queue.cc:1243] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:17.975179 15840 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.975646 15840 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:17.974956 19370 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.976747 19396 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:17.977658 15841 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:04:17.978065 15841 pending_rounds.cc:85] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 9
I20260709 13:04:17.978295 15841 pending_rounds.cc:107] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:17.978566 15841 raft_consensus.cc:2889] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:17.983852 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1089) took 3261 ms (client timeout 1950 ms). Trace:
W20260709 13:04:17.984370 15839 rpcz_store.cc:269] 0709 13:04:14.722606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.725194 (+  2588us) service_pool.cc:224] Handling call
0709 13:04:17.983665 (+3258471us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b540036db54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:17.983780 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5978317568}
I20260709 13:04:17.986186 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:17.986860 15159 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.987134 15846 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.973333 15844 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:17.988826 15846 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:17.989646 15079 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:17.991287 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.993578 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:17.996206 19399 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.996784 19399 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:17.997659 19078 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:17.998075 19078 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:17.998451 19353 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.998842 19078 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:17.976471 15840 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.000552 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:18.001128 15840 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.001907 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:18.005071 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 167440us
I20260709 13:04:18.008038 15841 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:18.007149 15844 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:04:18.005600 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 226323us
W20260709 13:04:18.009455 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.009807 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 92005.7us
I20260709 13:04:18.011345 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:18.012217 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.012645 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.013322 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:18.006034 19353 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:18.015264 15202 leader_election.cc:400] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:18.015635 15202 leader_election.cc:403] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:18.015813 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:18.016821 15084 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:18.017459 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1959) took 1162 ms. Trace:
I20260709 13:04:18.017737 15084 rpcz_store.cc:276] 0709 13:04:16.855350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.014866 (+1159516us) service_pool.cc:224] Handling call
0709 13:04:18.015175 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.017231 (+  2056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.017411 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:18.018440 15159 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:18.019677 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:18.020054 19347 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:18.040715 15126 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.859s (SENT)
I20260709 13:04:18.045408 19407 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:18.045632 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:18.045925 19407 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.047043 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:18.002661 19370 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.030036 19237 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:18.045658 19347 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.048422 19237 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:18.019078 19375 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:18.049449 19237 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.049669 19347 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.003126 19337 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:18.019878 15848 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:18.021622 15840 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:17.993146 19387 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:04:18.046341 19407 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.056241 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:18.056638 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:18.059945 15088 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:17.999178 19399 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.052985 19078 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:18.061331 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 50062.4us
I20260709 13:04:18.020309 19395 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:18.053154 19252 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:18.064472 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:18.062608 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:18.069530 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 1920) took 3615 ms (client timeout 2155 ms). Trace:
I20260709 13:04:18.064659 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:18.065083 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:18.070052 15233 rpcz_store.cc:269] 0709 13:04:14.454358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.062840 (+3608482us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:18.069446 (+  6606us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:18.071118 15084 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
W20260709 13:04:18.081912 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.078966 15088 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.072666 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:18.087896 15088 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:18.088238 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.072537 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:18.071913 19252 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.096053 19280 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:18.099090 15088 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:18.096872 19375 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:18.100467 19375 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.081532 19399 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:18.100775 15088 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.100988 19252 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:18.100857 19375 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.073418 19337 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:18.125396 15088 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:04:18.139412 15088 pending_rounds.cc:85] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 5
I20260709 13:04:18.139815 15088 pending_rounds.cc:107] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:04:18.140198 15088 raft_consensus.cc:2889] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:18.150527 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.151199 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.151540 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.156407 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 83912.7us
W20260709 13:04:18.159228 19408 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:04:18.164928 15086 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.015s	user 0.015s	sys 0.006s
I20260709 13:04:18.078223 19255 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.165480 15086 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:18.166272 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2575) took 1018 ms. Trace:
I20260709 13:04:18.167073 19255 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.124364 19211 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.167968 19211 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.170028 19211 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.170611 19399 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:18.166704 15086 rpcz_store.cc:276] 0709 13:04:17.148057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.148554 (+   497us) service_pool.cc:224] Handling call
0709 13:04:18.166176 (+1017622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:18.174999 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:18.176784 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:18.177851 15086 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:18.179687 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:18.184188 15088 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:18.185913 15088 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.188582 15088 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.189498 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:18.080557 19206 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.185917 19280 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:18.206596 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.211869 15088 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:04:18.139029 19337 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:18.222937 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.216964 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 176326us
W20260709 13:04:18.217811 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 215973us
W20260709 13:04:18.232546 15126 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.884s (SENT)
I20260709 13:04:18.233135 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:18.235365 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:18.199312 19253 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:18.193922 19399 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:04:18.222223 19337 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:18.208976 19211 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:04:18.209556 19255 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:04:18.228729 19353 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:18.253074 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 94846.9us
I20260709 13:04:18.239028 19387 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:18.238055 19394 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:18.244100 19253 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.243786 19395 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
W20260709 13:04:18.262970 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48679.2us
I20260709 13:04:18.261420 19253 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.272027 19211 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.270632 19414 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.274253 19387 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:18.275118 19410 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:18.297081 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61678.4us
I20260709 13:04:18.272070 19395 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:04:18.276154 19412 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:04:18.325716 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:18.293095 19414 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.280894 19211 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:18.276355 19407 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:18.346848 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.299052 19410 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.275002 19394 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:04:18.312752 19311 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:18.320827 19280 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:18.360832 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 128341us
W20260709 13:04:18.362440 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 106492us
I20260709 13:04:18.300802 19353 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.363998 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:18.358873 19416 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:18.357134 19311 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:18.351581 19410 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:18.369269 19423 raft_consensus.cc:2764] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:04:18.363984 19422 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:04:18.352144 19387 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:18.369692 19211 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.371017 19211 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.367210 19416 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.367736 19347 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:04:18.413605 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:18.410915 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:18.416021 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:18.416738 15232 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.390391 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:18.390748 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:18.386718 19416 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:18.393113 19311 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.396034 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:18.427297 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:18.428328 15085 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:18.396401 19280 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.397330 15084 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:18.402627 19375 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.433357 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:18.405154 19407 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.390026 19428 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:04:18.407083 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:18.437494 15088 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.410889 19422 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.411923 19378 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:18.422279 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.422148 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:18.425534 15231 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:18.432003 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:18.427448 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:18.403962 15086 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:18.433363 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:18.436944 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74286.5us
W20260709 13:04:18.445456 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.432235 19255 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:18.470165 15235 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:18.470363 15233 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:18.472056 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.472185 15126 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.155s (SENT)
I20260709 13:04:18.472568 15237 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:18.456275 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:18.433187 19399 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:18.435011 19347 consensus_queue.cc:1243] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:04:18.475490 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53452us
I20260709 13:04:18.476526 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:18.481940 15088 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:18.437517 19425 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:18.499908 15235 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:18.452184 19422 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.438428 19395 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:04:18.470796 19378 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:18.476123 19311 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:18.523896 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 96652.9us
I20260709 13:04:18.526108 15201 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:18.501277 19399 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:18.530259 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:18.531803 19425 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:18.532280 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.533282 15839 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.534158 19281 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:18.534569 19281 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:18.534960 19281 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:18.508684 19430 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:18.508147 19420 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:18.492616 19347 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:18.540210 15086 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:04:18.505731 19255 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:04:18.550737 15082 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.461s	user 0.016s	sys 0.000s
I20260709 13:04:18.551118 15082 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:18.551625 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1007) took 1478 ms. Trace:
I20260709 13:04:18.551980 15082 rpcz_store.cc:276] 0709 13:04:17.073403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.085164 (+ 11761us) service_pool.cc:224] Handling call
0709 13:04:18.551567 (+1466403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:18.554570 19423 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.521304 19427 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:04:18.545338 19430 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.547709 19420 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:18.554814 19382 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:04:18.546773 19437 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:18.384598 19414 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:18.571074 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 92166.7us
W20260709 13:04:18.560623 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74882.3us
I20260709 13:04:18.578430 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:18.580008 15840 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:18.580358 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:18.581226 15839 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:18.541965 19422 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.582878 15848 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.583310 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:18.547848 19252 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.590372 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:18.587993 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.588240 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:18.589255 15848 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:18.589499 15847 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.550073 19425 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:04:18.587960 15842 raft_consensus.cc:2955] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:18.589838 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:18.593922 15843 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:18.551474 19378 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:18.597424 15845 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:04:18.585618 19407 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.565097 19423 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.586525 19399 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.592520 19428 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.610603 19337 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.614059 19375 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.614547 19420 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.616396 19337 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.621852 19253 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.623078 19252 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:18.629681 19430 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.631712 19281 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.607123 19407 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.635730 19407 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:18.626606 19385 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:04:18.627115 19421 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.636722 19421 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:18.623588 19387 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:18.638242 19387 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.637990 19428 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:18.602517 19422 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.639194 19387 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.640136 19428 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.637849 19421 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.610509 19410 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.640515 19387 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:18.647544 19416 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.671377 19421 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.672437 19410 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.636651 19206 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.642334 19423 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.673136 19416 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.642030 19355 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:18.654233 19378 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:18.647230 19414 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.673864 19355 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:18.633536 19253 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:18.676157 19355 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.677696 19355 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:18.650449 19237 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:18.680680 19237 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:18.681571 19237 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.682343 19281 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:18.685731 19237 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:18.732666 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.734830 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.735898 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.736268 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.740015 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800281820 after lost signal to thread 13970
W20260709 13:04:18.941963 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800259c00 after lost signal to thread 15020
W20260709 13:04:18.942548 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800281960 after lost signal to thread 15034
W20260709 13:04:18.944505 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08003089c0 after lost signal to thread 15112
W20260709 13:04:18.754443 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.945900 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.762638 15085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:18.774712 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2583) took 1127 ms. Trace:
I20260709 13:04:18.948038 15085 raft_consensus.cc:606] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Received request to transfer leadership to TS 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:18.947990 15087 rpcz_store.cc:276] 0709 13:04:17.647168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.649582 (+  2414us) service_pool.cc:224] Handling call
0709 13:04:18.774230 (+1124648us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b540029f9d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:18.774489 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211471744}
I20260709 13:04:18.775874 15083 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:18.949203 19378 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.950031 19378 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.952394 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1971) took 1731 ms. Trace:
I20260709 13:04:18.948647 15085 raft_consensus.cc:614] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:18.952812 15083 rpcz_store.cc:276] 0709 13:04:17.220750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.232854 (+ 12104us) service_pool.cc:224] Handling call
0709 13:04:18.775328 (+1542474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.949518 (+174190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.952141 (+  2623us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b540029f9d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:18.952317 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894910720}
I20260709 13:04:18.737387 15080 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.955016 19371 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:18.956035 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1970) took 1860 ms. Trace:
I20260709 13:04:18.764520 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1973) took 1382 ms. Trace:
W20260709 13:04:18.956727 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.957458 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.956893 15081 rpcz_store.cc:276] 0709 13:04:17.382189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.762192 (+1380003us) service_pool.cc:224] Handling call
0709 13:04:18.763066 (+   874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.763587 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.763648 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:18.763747 (+    99us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:18.763912 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.764016 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.764219 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:18.760288 15228 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:18.804811 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2593) took 1032 ms. Trace:
W20260709 13:04:18.806483 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.774302 15843 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:18.766366 15840 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:18.761554 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.200423 15847 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.206240 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.207412 15846 raft_consensus.cc:2368] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:18.761556 19078 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.229137 15159 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:19.229765 19078 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.231279 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:18.840970 19454 ts_tablet_manager.cc:933] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
W20260709 13:04:18.841485 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 107410us
W20260709 13:04:19.234059 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.234571 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.234908 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.235935 15159 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:18.846271 19382 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:18.892642 19453 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.237193 19382 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.237629 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.237581 19453 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.238569 15839 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.239593 19382 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.240260 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:18.898023 19452 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.907397 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2591) took 1204 ms. Trace:
I20260709 13:04:18.956465 15080 rpcz_store.cc:276] 0709 13:04:17.095879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.096189 (+   310us) service_pool.cc:224] Handling call
0709 13:04:18.551588 (+1455399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.954565 (+402977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.955785 (+  1220us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540011a3d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:18.955956 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3346101120}
I20260709 13:04:18.952579 19378 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.241216 19452 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:18.960172 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:19.242419 15081 raft_consensus.cc:606] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:19.242863 15081 raft_consensus.cc:614] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:18.976198 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.244196 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.024782 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.006s	sys 0.005s
I20260709 13:04:18.993781 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 849) took 1405 ms. Trace:
W20260709 13:04:19.244731 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.008s	sys 0.005s
W20260709 13:04:19.027104 15203 connection.cc:256] RPC call timeout handler was delayed by -2.8901407349999317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
I20260709 13:04:19.245044 15845 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:04:19.245482 15203 connection.cc:256] RPC call timeout handler was delayed by -2.771340734999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 13:04:19.062233 15237 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.009s
W20260709 13:04:19.245838 15203 connection.cc:256] RPC call timeout handler was delayed by -2.5265407349999123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260709 13:04:19.050369 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.246158 15237 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:19.108561 15086 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:04:19.245092 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 812134us
W20260709 13:04:19.131533 15842 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.010s
I20260709 13:04:19.240460 19453 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:19.252890 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:19.253327 19453 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:19.253976 19378 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:19.254107 19453 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.256445 15845 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:19.176023 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.175525 15084 rpcz_store.cc:276] 0709 13:04:17.738850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.084941 (+346091us) service_pool.cc:224] Handling call
0709 13:04:18.396990 (+312049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.432051 (+ 35061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.771313 (+339262us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540011a8d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:18.771579 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716672768}
I20260709 13:04:19.185760 19237 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.246172 15203 connection.cc:256] RPC call timeout handler was delayed by -2.4139143849999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23090000000000002s for a response
W20260709 13:04:19.257781 15203 connection.cc:256] RPC call timeout handler was delayed by -1.9568407350000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2856s for a response
W20260709 13:04:19.258183 15203 connection.cc:256] RPC call timeout handler was delayed by -1.5124703159999626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28150000000000003s for a response
I20260709 13:04:19.221019 19412 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.225245 19356 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:04:19.241125 15079 rpcz_store.cc:276] 0709 13:04:17.702898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.924660 (+221762us) service_pool.cc:224] Handling call
0709 13:04:17.924974 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.990068 (+ 65094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.906848 (+916780us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:18.907214 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148298496}
W20260709 13:04:18.960235 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08000815e0 after lost signal to thread 15113
I20260709 13:04:19.244073 19452 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:19.116354 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.025s
W20260709 13:04:19.278587 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.000s	sys 0.026s
I20260709 13:04:19.134334 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.279364 15848 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:19.254655 19378 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:19.140098 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.159598 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.013s	sys 0.021s
W20260709 13:04:19.280637 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.013s	sys 0.021s
I20260709 13:04:19.244975 15229 rpcz_store.cc:276] 0709 13:04:17.587846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.588575 (+   729us) service_pool.cc:224] Handling call
0709 13:04:17.654774 (+ 66199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.665633 (+ 10859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.993170 (+1327537us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54000ad7d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:18.993463 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84179200}
W20260709 13:04:19.283102 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.283792 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.284713 19356 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:19.284865 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.285315 19356 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:19.285598 15841 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:19.286619 19356 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.288354 19356 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:19.232530 19078 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:19.290165 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea940 after lost signal to thread 15187
W20260709 13:04:19.292795 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800368160 after lost signal to thread 15188
W20260709 13:04:19.298183 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.301896 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:19.303606 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.311278 15846 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:19.289655 19378 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:19.302944 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.316711 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.314180 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:19.283146 19237 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.320604 19347 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.321187 15843 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:04:19.321316 19347 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:19.322307 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.323119 15843 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.323729 19347 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.324069 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.321599 15163 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:19.305032 19382 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.325138 15084 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:19.328683 19414 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:19.329011 19420 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.329187 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.329313 19414 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.330052 15846 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.330476 19416 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.331111 19416 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.331779 19414 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.313642 19371 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.332545 19371 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:19.333963 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:19.334524 19387 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:19.334872 19371 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.335342 19387 consensus_queue.cc:1243] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:19.336055 19387 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:19.336803 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:19.337139 19451 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:19.338174 19451 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:19.338518 19451 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:19.338840 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.339907 15081 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:19.339593 19451 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.337513 15154 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:19.341259 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:19.341917 15080 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:19.341954 19387 consensus_queue.cc:1243] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:19.330982 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.329622 19420 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.343863 15848 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:19.355598 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:19.358616 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:19.361145 15083 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.320607 19355 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.360242 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.351410 15846 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.390765 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.391085 19458 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:19.391343 19387 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:19.391678 15844 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:19.391857 19387 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:19.393181 19387 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.394070 19355 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:19.394930 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf080 after lost signal to thread 15261
W20260709 13:04:19.395324 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08003979c0 after lost signal to thread 15262
I20260709 13:04:19.317360 19412 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.396451 19355 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.397588 15158 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:19.434382 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:19.440532 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:19.441879 15163 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:19.442049 15079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:19.442780 15079 raft_consensus.cc:606] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:19.443265 15079 raft_consensus.cc:614] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.443303 19452 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.444114 19452 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.445724 15159 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:19.447800 15156 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:19.448163 19451 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:19.450176 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 146620us
W20260709 13:04:19.452340 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.397882 19454 tablet_copy_client.cc:323] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Beginning tablet copy session from remote peer at address 127.13.163.193:38845
I20260709 13:04:19.453802 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:19.459669 19416 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.391858 19458 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:19.456965 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.462054 19458 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:19.474578 19237 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:19.346148 19382 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:19.480072 15846 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:19.434278 19453 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:19.516445 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 224133us
I20260709 13:04:19.356365 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.392791 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:19.531728 15844 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:04:19.533252 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79614us
I20260709 13:04:19.534900 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.540884 19453 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:19.552455 19453 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:19.553226 19412 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.553718 19453 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.554673 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.546213 19387 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:19.555161 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.558153 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.558660 15843 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:19.561458 19237 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:19.562016 19237 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:19.563100 19382 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.562919 19214 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:19.568825 19452 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.569453 19214 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:19.581503 15841 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:19.582563 19214 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.584268 19214 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:19.562995 19237 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.569782 19452 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.587009 19452 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.599313 15848 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:19.600113 15841 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: [9.9-10.10]
I20260709 13:04:19.605047 15845 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
I20260709 13:04:19.606315 15083 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:19.607301 15086 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:04:19.606774 15845 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:19.608669 15844 raft_consensus.cc:2410] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:04:19.610435 15843 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:19.614148 15845 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:04:19.617147 15848 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:19.628197 15842 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:04:19.628618 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 174333us
W20260709 13:04:19.633229 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.634609 19387 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.635363 19387 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:19.637701 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.451375 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.638401 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.642091 19382 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.648506 19382 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.649109 19387 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.651010 19382 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:19.659010 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 117379us
I20260709 13:04:19.657996 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:19.668797 19471 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:19.669500 19471 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:19.670594 19471 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.678809 19472 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:04:19.679718 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:19.681555 19476 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:19.682263 19476 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:19.683312 19476 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.454870 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:19.687299 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 299334us
I20260709 13:04:19.688551 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:19.691040 19382 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:19.691742 19382 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:19.692157 19461 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:19.692890 19078 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:19.690295 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.693360 19078 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:19.693776 19378 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:19.694314 19078 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.695290 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.695678 19078 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:19.696094 19378 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:19.698022 19378 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:19.699200 19382 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:19.699577 19382 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:19.700421 19382 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.692718 19461 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.701474 19461 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:19.702167 19382 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:19.706122 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.706954 19214 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.707661 19214 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:19.712960 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.714022 19214 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.719614 15052 leader_election.cc:400] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:19.720032 15052 leader_election.cc:403] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:04:19.720582 19371 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:19.723861 19461 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.725427 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.726873 19421 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:19.724231 19382 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:19.727525 19421 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.728258 19382 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:19.729964 19421 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.730649 19421 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:19.733003 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:19.735381 19399 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:19.740860 15083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:19.741665 15083 raft_consensus.cc:606] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:19.742851 19420 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:19.743687 19078 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:19.744782 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:19.744728 19399 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:19.746393 19421 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:19.759699 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:19.750249 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.749346 19461 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.759948 19078 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:19.764343 19421 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.765041 19078 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.765096 19421 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.766710 19078 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:19.769250 19421 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.752574 19480 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:19.753364 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:19.772576 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:19.774574 19472 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:19.775560 19472 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:19.776086 19472 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:19.761524 19399 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.776897 19399 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.777550 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.777192 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.779897 19420 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:19.779611 19399 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.781425 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.777156 19472 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.779691 19452 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.784916 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.785740 15848 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:19.787370 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:19.788167 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.789425 19421 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:19.789934 19421 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.790289 15203 leader_election.cc:400] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8.
I20260709 13:04:19.790401 19421 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.791476 19399 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:19.791551 15203 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Timed out: Call waited in the queue past client deadline [suppressed 84 similar messages]
I20260709 13:04:19.792286 19399 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:19.799260 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.801550 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:19.803519 19480 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:19.803956 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:19.804636 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.805459 15228 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
I20260709 13:04:19.806622 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:19.807173 15236 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:19.808295 15237 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:19.809408 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.810058 15237 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:19.811986 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.813196 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.813797 19078 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.814301 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:19.822623 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.826077 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.823834 19421 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.826715 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.826934 15847 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:19.827327 15081 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:19.827448 14985 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.828930 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:19.830240 15081 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:04:19.830921 15848 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:19.831982 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:19.831127 14985 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.832917 19421 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:19.834664 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.835518 15088 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:19.825982 15086 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:19.838699 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:19.821149 19399 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:19.840493 19399 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.843194 19399 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:19.845687 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.853367 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.853482 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:19.830255 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080030d880 after lost signal to thread 13967
W20260709 13:04:19.854229 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800314c20 after lost signal to thread 13970
W20260709 13:04:19.855147 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080031bd60 after lost signal to thread 15011
W20260709 13:04:19.855532 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800208720 after lost signal to thread 15020
W20260709 13:04:19.855836 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800266940 after lost signal to thread 15034
I20260709 13:04:19.856123 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:19.856647 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.858055 19399 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.859031 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.859054 19399 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:19.862257 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:19.863008 15086 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:04:19.856388 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080018eac0 after lost signal to thread 15062
I20260709 13:04:19.863951 19399 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:19.864564 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080028bb00 after lost signal to thread 15112
I20260709 13:04:19.865454 15233 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:04:19.865831 15109 debug-util.cc:398] Leaking SignalData structure 0x7b080027bc20 after lost signal to thread 15187
W20260709 13:04:19.866103 15109 debug-util.cc:398] Leaking SignalData structure 0x7b08000abfa0 after lost signal to thread 15188
I20260709 13:04:19.866577 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:19.867188 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800412e80 after lost signal to thread 15261
W20260709 13:04:19.867463 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800412e60 after lost signal to thread 15262
I20260709 13:04:19.867781 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:19.868661 15109 debug-util.cc:398] Leaking SignalData structure 0x7b0800412580 after lost signal to thread 15872
I20260709 13:04:19.810870 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.868822 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:19.870652 19378 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.881815 15233 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:19.871922 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.068809 15237 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:20.070344 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.071125 15237 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:20.072233 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:19.877103 19214 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:19.880157 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:20.171866 19214 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:20.172216 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 292142us
I20260709 13:04:19.879680 19451 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:04:19.887990 19378 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:19.872345 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.317471 19378 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:19.815335 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:19.879142 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80021.6us
W20260709 13:04:20.096939 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 221044us
I20260709 13:04:19.877825 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:20.102712 19468 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.509s	user 0.003s	sys 0.005s
W20260709 13:04:20.329725 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 451943us
I20260709 13:04:20.092458 15237 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:20.330428 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:20.331053 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.331283 19356 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.331902 15237 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:20.333222 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1072) took 1078 ms. Trace:
I20260709 13:04:20.333614 15840 rpcz_store.cc:276] 0709 13:04:19.254768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.277802 (+ 23034us) service_pool.cc:224] Handling call
0709 13:04:19.298646 (+ 20844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.356717 (+ 58071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.360135 (+  3418us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.438367 (+ 78232us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.543314 (+104947us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:19.543371 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.543434 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.102120 (+558686us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:20.332906 (+230786us) raft_consensus.cc:1685] finished
0709 13:04:20.332931 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.333141 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16375,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":2900,"spinlock_wait_cycles":1109985664,"thread_start_us":56670,"threads_started":1,"wal-append.queue_time_us":113335}
I20260709 13:04:20.335889 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:20.336536 15840 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:20.109673 15229 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:20.340149 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.340817 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:20.341969 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.342729 15229 raft_consensus.cc:2376] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10. Re-sending same reply.
I20260709 13:04:20.343634 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.344226 15229 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:20.110496 19480 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.113356 19481 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.345129 19480 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.345366 19481 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.347635 19480 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.348436 19481 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.114567 19487 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.119401 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.120467 15158 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:20.120525 15154 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:20.121644 15163 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:20.121996 15162 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:20.122503 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.122996 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:20.123454 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.351115 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.351202 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.351756 15163 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:20.351884 15159 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:20.352371 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.123636 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.352698 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:20.353340 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.124423 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.353502 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:20.351251 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:20.352761 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:20.132825 19470 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.485s	user 0.005s	sys 0.004s
I20260709 13:04:20.138123 15232 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.355962 19355 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:20.354255 15231 raft_consensus.cc:2376] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11. Re-sending same reply.
I20260709 13:04:20.148564 19421 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.150365 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.357255 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:20.357648 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.358232 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.357546 19421 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.358383 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:20.358711 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.359066 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.359359 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.359499 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:20.159118 19483 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.380s	user 0.005s	sys 0.000s
I20260709 13:04:20.360044 15231 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:20.360265 19421 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:20.153787 19458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.006s	sys 0.008s
W20260709 13:04:20.361003 19458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.006s	sys 0.008s
I20260709 13:04:20.166658 19382 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:20.361858 19382 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:20.362138 15161 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:20.363257 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.362920 19382 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.363914 15161 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:20.364459 19382 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.172950 19214 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:20.187625 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800391740 after lost signal to thread 15872
W20260709 13:04:20.205989 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.219317 19452 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.216801 15080 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.874s	user 0.006s	sys 0.015s
I20260709 13:04:20.370874 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:20.223739 19414 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:20.229637 19471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.013s
I20260709 13:04:20.234305 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:20.244549 14983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 321) took 37301 ms (client timeout 15000 ms). Trace:
I20260709 13:04:20.247570 14985 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 6, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 6 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:20.253167 19463 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.724s	user 0.009s	sys 0.000s
I20260709 13:04:19.878687 19078 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:20.258200 19465 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.703s	user 0.000s	sys 0.008s
I20260709 13:04:19.880009 19412 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.381s
W20260709 13:04:20.266245 19482 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.463s	user 0.000s	sys 0.001s
W20260709 13:04:20.273784 19466 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.746s	user 0.007s	sys 0.001s
I20260709 13:04:20.274962 15086 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:20.386999 19414 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:20.387514 14985 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 4, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 4 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:20.388527 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.275637 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:20.372323 19471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.007s	sys 0.013s
I20260709 13:04:20.388710 19414 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:20.377401 19452 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.278647 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.279042 19494 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:20.392202 15081 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:20.279541 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.280301 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.281937 15097 tablet_copy_service.cc:140] P 12daa67457954a0cab27e32bf85fa115: Received BeginTabletCopySession request for tablet ff3342a5b2ad49e2b252d7cd47b9851d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 ({username='slave'} at 127.0.0.1:56652)
I20260709 13:04:20.282418 19451 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:20.399396 15097 tablet_copy_service.cc:161] P 12daa67457954a0cab27e32bf85fa115: Beginning new tablet copy session on tablet ff3342a5b2ad49e2b252d7cd47b9851d from peer 3c28be6f86ea4fb6b79a47d3999f5a82 at {username='slave'} at 127.0.0.1:56652: session id = 3c28be6f86ea4fb6b79a47d3999f5a82-ff3342a5b2ad49e2b252d7cd47b9851d
I20260709 13:04:20.399585 19451 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:20.399783 15156 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.400553 19451 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.401507 15156 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:20.402145 19451 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:20.402907 15156 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.404345 15156 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:20.283213 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:20.283546 19484 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.485s	user 0.001s	sys 0.000s
I20260709 13:04:20.405167 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:04:20.405303 15202 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.653s (SENT)
W20260709 13:04:20.287235 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.405828 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:20.406185 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.406627 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.407387 19458 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:20.300307 15230 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:04:20.307139 19476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.013s	sys 0.005s
I20260709 13:04:19.880627 19387 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.429s
W20260709 13:04:20.311379 19475 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.633s	user 0.009s	sys 0.000s
I20260709 13:04:20.320684 15843 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.320930 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.321219 15841 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:04:20.321355 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:20.416702 15848 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.321409 19378 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:20.322491 15051 leader_election.cc:336] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.850s (SENT)
I20260709 13:04:20.323203 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.323211 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:20.323519 19399 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:20.326233 19237 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.331856 19356 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.108511 15233 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:20.108636 15236 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7.
W20260709 13:04:20.337855 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 162726us
I20260709 13:04:20.338923 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.348649 15228 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:04:20.114013 19479 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.356s	user 0.002s	sys 0.005s
I20260709 13:04:20.125594 15157 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:20.354653 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:20.428242 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 73694.4us
I20260709 13:04:20.126070 15087 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:04:20.353614 15163 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:04:20.428673 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.349421 19487 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.164182 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.360944 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.176769 19416 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:20.365527 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.357255 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:20.366133 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.355973 15162 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:04:20.369320 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:20.370047 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08003e4ec0 after lost signal to thread 18911
I20260709 13:04:20.370347 15158 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:20.371203 15080 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:20.371753 15154 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:20.433768 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1033) took 1096 ms. Trace:
I20260709 13:04:20.434273 15080 rpcz_store.cc:276] 0709 13:04:19.337403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.341002 (+  3599us) service_pool.cc:224] Handling call
0709 13:04:20.433691 (+1092689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.434806 15154 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:20.435637 15083 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:20.436314 15154 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:20.438060 15161 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:20.440228 15237 raft_consensus.cc:2368] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:20.440722 15154 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:04:20.355679 19453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.017s	sys 0.018s
W20260709 13:04:20.385042 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53764us
I20260709 13:04:20.394955 15086 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:20.394335 15085 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:20.397318 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1031) took 1060 ms. Trace:
I20260709 13:04:20.398471 15079 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:20.399174 19452 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:20.295977 19473 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.622s	user 0.000s	sys 0.007s
I20260709 13:04:20.460242 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.464612 15229 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:20.373039 19474 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:20.409216 14985 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:20.413869 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 290673us
I20260709 13:04:20.465653 19474 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:20.416456 15841 pending_rounds.cc:85] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 8
I20260709 13:04:20.466336 19474 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:20.414649 19476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.014s	sys 0.005s
I20260709 13:04:20.467465 19476 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:20.407624 19478 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.469273 19478 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.422958 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.415143 19355 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:20.473546 19355 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.426368 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.426674 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1077) took 1085 ms. Trace:
I20260709 13:04:20.411723 19078 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:20.476477 19078 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.476997 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.476179 15844 rpcz_store.cc:276] 0709 13:04:19.341361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.610850 (+269489us) service_pool.cc:224] Handling call
0709 13:04:19.611576 (+   726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.613563 (+  1987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.613590 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.614129 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.668160 (+ 54031us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:19.668203 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.668294 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.310691 (+642397us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:20.426366 (+115675us) raft_consensus.cc:1685] finished
0709 13:04:20.426394 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.426612 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39170,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":920833920,"thread_start_us":53606,"threads_started":1,"wal-append.queue_time_us":63312}
I20260709 13:04:20.478482 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.479457 19078 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:20.431770 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.430570 15235 raft_consensus.cc:2376] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 7. Re-sending same reply.
W20260709 13:04:20.481576 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.482059 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.482757 15235 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:20.482825 19078 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.483834 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.483625 19078 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:20.484479 15235 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:04:20.453925 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 130830us
I20260709 13:04:20.485556 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.484570 19478 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.485734 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.458426 15097 tablet_copy_source_session.cc:215] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:20.490316 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.407087 15156 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:20.449999 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.428112 19356 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.466457 15841 pending_rounds.cc:107] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:20.494639 15841 raft_consensus.cc:2889] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:20.418639 15234 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:20.353605 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.495666 15160 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:20.479480 15236 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 8: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:20.497103 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.498045 19478 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:20.498118 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.498592 19478 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.499060 19478 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.499203 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.500363 15236 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:04:20.482230 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.448199 15232 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.451272 15084 rpcz_store.cc:276] 0709 13:04:19.336666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.337923 (+  1257us) service_pool.cc:224] Handling call
0709 13:04:19.338244 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.338608 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.338635 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.339156 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.456234 (+117078us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:19.456282 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.458444 (+  2162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.252124 (+793680us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:20.393291 (+141167us) raft_consensus.cc:1685] finished
0709 13:04:20.393321 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.397230 (+  3909us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17367,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":109,"spinlock_wait_cycles":1659892480,"thread_start_us":99464,"threads_started":1,"wal-append.queue_time_us":189225}
W20260709 13:04:20.501236 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.462968 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.502039 15237 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:20.509497 15232 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:20.429185 19371 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.461632 19453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.018s	sys 0.018s
I20260709 13:04:20.437949 19461 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:20.438033 19416 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:20.526391 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 60162.3us
I20260709 13:04:20.432895 19471 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.514662 15232 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:04:20.510808 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:20.450035 19472 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.511557 15163 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
W20260709 13:04:20.521191 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 69942.4us
I20260709 13:04:20.536051 15163 pending_rounds.cc:85] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 7
I20260709 13:04:20.522063 15230 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:20.536470 15163 pending_rounds.cc:107] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:20.522688 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:20.536895 15163 raft_consensus.cc:2889] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:20.537158 15235 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:20.530540 15232 pending_rounds.cc:85] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 7
I20260709 13:04:20.538298 15232 pending_rounds.cc:107] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:20.510877 19420 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.538589 15232 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:20.539028 15232 pending_rounds.cc:107] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:20.539359 15232 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:20.428283 19237 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.545910 15086 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:20.529538 19416 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:20.550307 15228 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:20.551679 15234 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:04:20.530400 19471 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:20.554606 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:20.558828 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 236347us
I20260709 13:04:20.558591 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1134) took 1272 ms. Trace:
I20260709 13:04:20.561529 15086 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:20.561901 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:20.566030 15848 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:20.549731 19498 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:04:20.524329 19371 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.568831 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:20.569326 15230 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:20.570883 15233 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:04:20.552224 19078 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.552619 19494 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.558627 19471 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.548099 19420 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:20.579270 15051 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.906s (SENT)
I20260709 13:04:20.583851 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.586412 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:20.559635 15846 rpcz_store.cc:276] 0709 13:04:19.286529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.547771 (+261242us) service_pool.cc:224] Handling call
0709 13:04:19.548107 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.548479 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.548507 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.549021 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.667442 (+118421us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:19.667473 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.667527 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.295183 (+627656us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:20.555615 (+260432us) raft_consensus.cc:1685] finished
0709 13:04:20.555637 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.558525 (+  2888us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1356,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6764,"spinlock_wait_cycles":918776704,"thread_start_us":118024,"threads_started":1,"wal-append.queue_time_us":124609}
I20260709 13:04:20.581852 15230 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:20.588289 15083 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:20.560858 19487 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:20.589316 15051 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.950s (SENT)
I20260709 13:04:20.589370 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:20.590389 15086 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:04:20.556716 19477 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.595714 19497 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.596717 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.597067 19501 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:20.597640 19501 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.598006 19501 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:20.603856 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 52687.8us
I20260709 13:04:20.560822 19414 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.542770 19472 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.611219 15084 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:20.612032 15085 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:04:20.615844 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 179851us
I20260709 13:04:20.616501 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.578085 19412 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.372617 14983 rpcz_store.cc:269] 0709 13:03:42.943033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.954543 (+ 11510us) service_pool.cc:224] Handling call
0709 13:04:20.244286 (+37289743us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:15.097870 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:15.113651 (+ 15781us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:15.113703 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:04:15.113725 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:15.114920 (+  1195us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:15.115020 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:15.115060 (+    40us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:15.117392 (+  2332us) tablet.cc:701] Row locks acquired
0709 13:04:15.117459 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:04:15.117714 (+   255us) write_op.cc:270] Start()
0709 13:04:15.117890 (+   176us) write_op.cc:276] Timestamp: P: 1783602255117618 usec, L: 0
0709 13:04:15.117926 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:04:15.118915 (+   989us) log.cc:844] Serialized 4410 byte log entry
0709 13:04:16.730375 (+1611460us) op_driver.cc:464] REPLICATION: finished
0709 13:04:16.805350 (+ 74975us) write_op.cc:301] APPLY: starting
0709 13:04:16.805424 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:16.819645 (+ 14221us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:16.819773 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:16.935353 (+115580us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:16.935780 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:04:16.935846 (+    66us) write_op.cc:312] APPLY: finished
0709 13:04:16.936354 (+   508us) log.cc:844] Serialized 105 byte log entry
0709 13:04:16.936586 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:16.936781 (+   195us) write_op.cc:454] Released schema lock
0709 13:04:16.943631 (+  6850us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:16.944193 (+   562us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:16.944232 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":764374,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":74718,"apply.run_cpu_time_us":20029,"apply.run_wall_time_us":138758,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43953,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":1,"lbm_read_time_us":960,"lbm_reads_lt_1ms":8,"num_ops":11,"prepare.queue_time_us":111075,"prepare.run_cpu_time_us":19558,"prepare.run_wall_time_us":121266,"raft.queue_time_us":46048,"raft.run_cpu_time_us":43977,"raft.run_wall_time_us":75007,"replication_time_us":1612277,"thread_start_us":225916,"threads_started":4,"wal-append.queue_time_us":1501115,"wal-append.run_cpu_time_us":13683,"wal-append.run_wall_time_us":1319838}]]}
I20260709 13:04:20.625838 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.627483 15088 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:20.606412 19378 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:20.610957 19214 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.596386 19497 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.645454 15235 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:20.640139 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.642665 19382 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:20.615367 19414 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.615064 19477 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.642030 19237 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.603092 19474 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.639937 19499 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:20.676555 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 146398us
I20260709 13:04:20.645757 19356 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.658205 19472 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:20.658867 19458 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:20.661115 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.673061 15085 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:04:20.674098 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 223206us
I20260709 13:04:20.674603 15084 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:20.635469 19412 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.677265 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:20.679345 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:20.654901 19214 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.594020 19494 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.683991 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.686571 15159 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:20.698944 15083 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:20.704897 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:20.711586 19356 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.712286 19356 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.712775 19356 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.681515 19458 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:20.698693 19214 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.685930 19412 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.682664 19503 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:20.737185 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.738015 15163 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:20.738849 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.739707 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.741180 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.741951 15079 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:04:20.734360 19487 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:20.742676 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.742916 19487 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.743997 19474 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.745780 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.747357 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.749138 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.748097 15163 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
W20260709 13:04:20.750808 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.749773 15162 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:20.683681 19514 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:04:20.741371 19472 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.756450 19472 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.756872 19472 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.756723 19420 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.757550 19420 raft_consensus.cc:2764] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:04:20.757692 19378 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:20.758148 19378 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:20.758662 19478 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.759006 19378 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.759827 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:20.760365 19378 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:20.760970 19515 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:20.761202 19458 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.761539 19515 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.762802 19458 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.763787 19453 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.764209 19453 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.764369 19515 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.764577 19453 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.769014 15088 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.744745 19474 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:20.777107 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.777823 15160 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:20.754462 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.746359 19518 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:04:20.784955 19356 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.777413 19474 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:20.780632 19382 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:04:20.785951 19371 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.745563 19237 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:04:20.761106 19420 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:20.789471 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:20.797955 15844 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.781088 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:20.776765 19505 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:20.745440 19416 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:20.805035 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.827345 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 122544us
I20260709 13:04:20.806385 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:20.830447 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.796375 19474 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.795964 19478 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.818346 19505 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.828994 19371 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.837634 19371 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.840286 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:20.841681 19524 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:20.842218 19524 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:20.842689 19524 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:20.847026 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.829933 19420 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:20.835579 19078 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:20.838056 19487 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.837253 19505 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:20.838083 19371 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:20.857821 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 196773us
I20260709 13:04:20.866107 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.869441 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:20.869781 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:20.870690 19451 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:20.871748 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.872673 19378 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:20.874679 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:20.871851 19382 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:20.883329 15079 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:20.884986 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.886741 19522 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:20.887185 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.887352 15083 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:04:20.887773 15083 pending_rounds.cc:85] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:04:20.888013 15083 pending_rounds.cc:107] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:20.888273 15083 raft_consensus.cc:2889] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:20.892181 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.885056 19521 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:04:20.895596 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.896430 15084 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:20.887367 19522 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:20.898089 19522 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:20.902245 19501 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.903694 15086 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:20.905251 19501 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.910729 15160 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:04:20.913439 19471 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.876736 19382 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.925314 19382 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:20.861586 19519 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:20.875172 19477 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.909222 19420 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.937436 19494 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.938112 19214 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.940274 19214 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.927902 19472 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.946450 15088 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:20.962633 15844 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:20.965637 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49425.3us
I20260709 13:04:20.965790 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:21.040378 15084 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:20.968000 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 108495us
I20260709 13:04:20.945263 19513 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:20.975859 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:20.977103 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.985419 19497 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:20.987226 19505 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.944610 19453 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:20.970181 19399 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.967578 19499 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:20.996131 15163 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:20.995935 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:20.996001 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:20.996636 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.006491 19371 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.065115 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.065447 19371 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.944754 19474 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.068275 19474 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:20.971856 19487 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:04:21.014963 19537 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.016189 19414 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:21.017185 19416 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.011765 19382 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:21.020311 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.072021 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.074725 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:21.020937 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:21.026221 15258 debug-util.cc:398] Leaking SignalData structure 0x7b0800409420 after lost signal to thread 13970
W20260709 13:04:21.077994 15258 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbde0 after lost signal to thread 15020
W20260709 13:04:21.078474 15258 debug-util.cc:398] Leaking SignalData structure 0x7b080008d9c0 after lost signal to thread 15034
W20260709 13:04:21.079733 15258 debug-util.cc:398] Leaking SignalData structure 0x7b080009a2e0 after lost signal to thread 15112
I20260709 13:04:21.029502 19519 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:21.030648 19496 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:21.030553 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 156053us
I20260709 13:04:21.031985 19501 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.082122 19501 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.081561 19496 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:21.085572 19496 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:21.089003 15088 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:21.089206 19537 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.091675 19537 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.092223 19537 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.093899 19501 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:21.041559 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.051064 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.000272 19524 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:21.047286 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 196521us
I20260709 13:04:21.097539 19454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3342a5b2ad49e2b252d7cd47b9851d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.099004 19524 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.081458 15161 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:21.100127 19524 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:21.101819 19519 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:21.095904 19382 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.034000 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.081470 19534 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:04:21.105365 19382 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:21.106156 19382 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.109134 19382 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.105206 19519 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:04:21.107532 15258 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad7e0 after lost signal to thread 15187
W20260709 13:04:21.111318 15258 debug-util.cc:398] Leaking SignalData structure 0x7b08001f35e0 after lost signal to thread 15188
I20260709 13:04:21.111150 19476 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:21.112531 15258 debug-util.cc:398] Leaking SignalData structure 0x7b0800381ac0 after lost signal to thread 15261
W20260709 13:04:21.112872 15258 debug-util.cc:398] Leaking SignalData structure 0x7b080040c120 after lost signal to thread 15262
W20260709 13:04:21.114127 15258 debug-util.cc:398] Leaking SignalData structure 0x7b08003776c0 after lost signal to thread 15872
W20260709 13:04:21.114462 15258 debug-util.cc:398] Leaking SignalData structure 0x7b0800338a40 after lost signal to thread 15873
W20260709 13:04:21.105201 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71280.7us
I20260709 13:04:21.116530 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.117619 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.118511 19371 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.118948 19526 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:21.119120 19371 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:21.119745 19526 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:21.120021 19371 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.092671 19537 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.123499 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.123492 19501 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:21.124418 15161 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.126551 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.106917 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:21.120237 19526 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:21.129906 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:21.130816 19476 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.131006 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:21.131381 19476 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:21.131880 19476 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.107813 19534 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:21.137176 19474 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.137801 19474 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.138254 19474 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.140049 15161 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:21.145401 19519 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:21.146195 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:21.147428 19519 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:21.148367 19416 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:21.151896 19476 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.153075 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.161149 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.161791 15084 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:21.162196 15083 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:21.156221 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.160108 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.163265 15086 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.168069 19476 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:21.169747 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70554.2us
W20260709 13:04:21.171073 19545 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:04:21.175446 15232 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:21.177121 15229 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:04:21.178187 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.179966 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.180218 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.181461 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.182329 15236 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:21.183822 19548 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:21.185761 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.186406 15228 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.185456 19382 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.190101 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.190971 15229 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:21.192472 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.184391 19548 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.194711 15232 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:04:21.203646 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.196460 19548 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.207662 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:21.208276 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:21.213128 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 151808us
W20260709 13:04:21.215425 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67332us
I20260709 13:04:21.050995 19378 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:21.217432 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 44711.8us
I20260709 13:04:21.218524 15088 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:04:21.180630 15086 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:21.225430 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:21.230495 15843 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:21.246955 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 184235us
I20260709 13:04:21.214427 19498 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:21.286988 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:21.290840 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.291805 15237 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:21.298784 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.299098 15201 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff3342a5b2ad49e2b252d7cd47b9851d. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.301703 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 81027.1us
I20260709 13:04:21.302539 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:21.308272 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:21.309571 19513 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.310071 19237 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:21.306209 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.208598 19399 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.229226 19487 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:04:21.190166 19382 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.314695 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:21.314347 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:21.282734 19471 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:21.325299 15842 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:21.315269 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:21.226989 19458 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:04:21.315053 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:21.316668 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 344163us
I20260709 13:04:21.322167 15228 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:21.332571 15163 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:21.333962 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:21.336042 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.337366 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.338039 15086 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.343963 15810 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:21.325906 19416 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.344631 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.351814 15159 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.323166 19378 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:21.351217 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 125460us
I20260709 13:04:21.347993 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:21.316421 19498 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:21.365466 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:21.369160 15086 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:21.339061 19513 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:21.338192 19472 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:21.390576 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.357547 19420 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:21.337306 19518 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:21.321462 19526 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
W20260709 13:04:21.412936 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50530.6us
I20260709 13:04:21.375659 19378 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:21.324869 19499 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.350881 19548 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:04:21.437743 19472 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.437501 19451 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:21.382997 19513 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:21.323431 19537 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.407464 19515 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:21.364933 19477 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:04:21.450367 19399 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:21.403311 19214 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.418790 19496 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:21.453061 19540 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:04:21.476863 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:21.447774 15843 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:21.492177 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 141094us
W20260709 13:04:21.471689 19551 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
W20260709 13:04:21.524129 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 176211us
W20260709 13:04:21.551815 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 198862us
I20260709 13:04:21.556717 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:21.557333 19501 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:21.558298 19496 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.558966 19563 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.559221 15160 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:21.559463 19563 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.560379 19399 raft_consensus.cc:2764] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:04:21.549866 19453 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:21.561559 19563 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:21.562175 19563 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:21.562551 19563 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:21.563478 19563 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.555872 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:21.567927 19501 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:21.568492 19501 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:21.569481 19501 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.572523 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:21.573997 19371 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.574383 19496 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.575268 19496 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.575642 19496 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.576306 19371 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.577020 19214 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.579382 19214 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:21.579943 19487 consensus_queue.cc:1243] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:21.498718 19548 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:04:21.583747 19453 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:21.584128 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 107388us
I20260709 13:04:21.584327 19453 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:21.585390 19453 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.586601 19568 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:21.587159 19568 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:21.588065 19568 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.589007 19214 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:21.589988 19550 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:21.591480 19498 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.595992 19537 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:21.611301 15159 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:21.614862 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.479686 19513 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:21.496104 19475 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:21.621129 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.625170 15162 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:21.625589 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.626415 15163 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 7.
W20260709 13:04:21.624698 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 410769us
I20260709 13:04:21.493155 19515 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:21.549073 19505 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:21.678936 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.679821 15162 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:21.680313 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:21.628602 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:21.677704 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.688875 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.690254 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.676354 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:21.697755 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 395299us
W20260709 13:04:21.699431 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 70954.4us
I20260709 13:04:21.699623 19564 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:21.700759 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:21.701139 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:21.628530 19371 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.499823 19535 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.134s
I20260709 13:04:21.712508 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:21.706640 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.707007 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.628060 19477 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.710368 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:21.711699 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.717378 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:21.711988 15087 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:04:21.720532 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.680281 19515 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:21.725739 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:21.758244 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.772776 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:21.775943 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 303674us
I20260709 13:04:21.701555 19503 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:21.781243 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.787384 15229 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.790227 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.791000 15237 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.791751 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:21.792659 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.712139 19537 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:21.793287 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:21.798602 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.801911 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.807896 19382 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.808652 19382 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.819170 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.819794 15159 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.827916 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:21.831136 15081 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.838563 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.839421 15231 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 7: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:21.776875 19476 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:04:21.798776 19514 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:21.862555 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.877592 15233 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.878172 15159 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:21.863827 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:21.713083 19378 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.726559 19477 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:21.719069 19371 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:21.803305 19503 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:21.869596 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.879884 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:21.881490 15229 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:21.801450 19356 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:04:21.914582 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:21.897536 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 102467us
I20260709 13:04:21.915539 15159 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:21.917490 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:21.919207 15159 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:21.916096 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 196252us
I20260709 13:04:21.929781 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.937806 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.938201 15159 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:21.939004 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.939445 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:21.939899 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.945751 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:21.943070 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 139822us
I20260709 13:04:21.947782 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:21.948602 15082 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:21.935483 19474 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.797s	user 0.013s	sys 0.000s
I20260709 13:04:21.955322 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:21.928994 19501 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:21.963351 15233 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:21.965806 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.921625 19573 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:21.938617 19526 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:21.976068 15237 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:21.977279 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:21.889365 19524 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:04:21.944227 19505 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:21.966648 19536 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:04:22.066154 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.067080 15229 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:22.049751 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:22.061355 19501 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:22.051195 15233 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:04:21.961143 19472 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:21.965962 19214 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:21.962785 19559 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:21.888979 19554 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.369s
I20260709 13:04:22.114892 15231 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:22.075191 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 112975us
I20260709 13:04:22.076408 19452 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:22.132308 19452 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:22.084832 19564 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:04:22.135099 19452 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.140069 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:22.144861 15081 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:21.966531 19474 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.146682 15155 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:22.070545 19078 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.061919 19524 consensus_queue.cc:1243] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:22.159724 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:22.157212 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:22.162271 15082 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.157992 15233 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.163161 15233 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:22.164680 19518 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:22.070072 19514 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.177470 19577 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:04:22.177198 19514 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.179883 19514 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.181687 15081 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:22.182895 19078 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.140161 19501 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:22.106961 19356 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:21.969226 19558 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.406s
I20260709 13:04:21.971347 19578 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.173s
I20260709 13:04:22.165328 19518 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:22.173679 19556 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.730s	user 0.014s	sys 0.006s
W20260709 13:04:22.174338 19566 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:04:22.174129 19584 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.210s
W20260709 13:04:22.175462 19567 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:04:22.175772 19592 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.149302 19591 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:04:22.189178 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 227205us
I20260709 13:04:22.109679 19214 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:22.112452 19559 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:22.180523 19514 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:22.190886 15233 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:22.208124 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.074318 19505 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:22.167042 19472 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:22.247989 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:22.259749 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.249095 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:22.260929 15848 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:22.261056 15843 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.252254 15844 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:22.107698 19503 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:22.247169 19478 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:22.258224 19595 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:22.253211 19559 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.196460 19356 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.323164 15233 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.335996 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:22.337144 15083 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:22.349208 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.307703 19478 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:22.354811 19420 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.010s	sys 0.006s
W20260709 13:04:22.355196 19453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.008s	sys 0.009s
I20260709 13:04:22.365705 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:22.247838 19592 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.314241 19595 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:22.323943 19477 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.310921 19537 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:22.396922 15162 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:22.397476 15846 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:22.285684 19514 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.387583 15842 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:22.401924 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.400033 15843 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:22.402410 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.019s	sys 0.018s
W20260709 13:04:22.403445 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.020s	sys 0.019s
I20260709 13:04:22.283891 19472 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.388238 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1068) took 1052 ms. Trace:
I20260709 13:04:22.348762 19371 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.405076 15083 pending_rounds.cc:85] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
W20260709 13:04:22.387980 19453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.008s	sys 0.009s
W20260709 13:04:22.403914 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 680929us
I20260709 13:04:22.411056 15083 pending_rounds.cc:107] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 8.8
W20260709 13:04:22.381923 19420 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.011s	sys 0.006s
I20260709 13:04:22.409451 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 885) took 1211 ms. Trace:
I20260709 13:04:22.311265 19572 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:04:22.406286 15088 rpcz_store.cc:276] 0709 13:04:21.335615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.335783 (+   168us) service_pool.cc:224] Handling call
0709 13:04:21.336181 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.347861 (+ 11680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.347908 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.348514 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.547075 (+198561us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:21.547134 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.547235 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.387925 (+840690us) raft_consensus.cc:1685] finished
0709 13:04:22.387954 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.388158 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10573,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10093,"thread_start_us":149978,"threads_started":1,"wal-append.queue_time_us":449477}
I20260709 13:04:22.395911 19571 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:22.432412 15848 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:22.430917 15083 raft_consensus.cc:2889] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:22.431203 15236 rpcz_store.cc:276] 0709 13:04:21.197969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.199517 (+  1548us) service_pool.cc:224] Handling call
0709 13:04:21.199883 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.200120 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.200148 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.200735 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.241291 (+ 40556us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:21.241324 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.242130 (+   806us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.157708 (+915578us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:22.389122 (+231414us) raft_consensus.cc:1685] finished
0709 13:04:22.389150 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.409377 (+ 20227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23086336,"thread_start_us":40040,"threads_started":1,"wal-append.queue_time_us":242084}
I20260709 13:04:22.439677 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:22.440565 15841 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:22.416963 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:22.452519 15233 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:22.343158 19214 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:22.484694 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.485659 15842 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:22.488101 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:22.489995 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 72845.9us
W20260709 13:04:22.491696 19594 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:04:22.493999 19416 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.136s	user 0.010s	sys 0.001s
W20260709 13:04:22.494347 19597 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:04:22.494632 19416 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:22.495368 15161 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:22.497267 19537 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.490584 19596 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 13:04:22.477171 19598 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:22.505213 15841 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:04:22.511956 15844 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:22.517990 19472 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.519533 19592 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.518034 15081 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:04:22.521435 15081 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:04:22.395999 19478 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.527969 19598 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.537233 19478 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:22.538080 19496 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.962s	user 0.016s	sys 0.001s
I20260709 13:04:22.538619 19598 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.538511 19496 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.539445 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.540019 15841 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:22.509045 19595 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.541842 19595 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:22.541955 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:22.541913 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:22.543236 19513 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.874s	user 0.007s	sys 0.002s
I20260709 13:04:22.543464 19501 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:22.543881 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:22.533661 15843 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:22.543972 19501 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:22.545387 19501 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:22.505677 19593 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:04:22.506400 19590 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:04:22.505323 19214 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.543694 19513 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.550536 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:22.540673 19496 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.524789 19518 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.552549 19371 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:22.583370 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:22.600512 15847 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:22.582118 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:22.574493 19579 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:04:22.553004 19477 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:22.610792 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:22.574762 19598 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.577075 19596 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:22.580168 19587 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:22.416217 19505 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:22.615284 19518 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:22.638480 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 1989) took 1014 ms. Trace:
I20260709 13:04:22.639067 15080 rpcz_store.cc:276] 0709 13:04:21.623880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.624005 (+   125us) service_pool.cc:224] Handling call
0709 13:04:21.678390 (+ 54385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.678716 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.678743 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.679172 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.931715 (+252543us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:21.931773 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.931863 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.638173 (+706310us) raft_consensus.cc:1685] finished
0709 13:04:22.638213 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.638401 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2682,"raft.queue_time_us":80,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":12739,"thread_start_us":154961,"threads_started":1,"wal-append.queue_time_us":495559}
I20260709 13:04:22.591475 19526 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:04:22.644629 15082 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:22.630271 19356 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.624972 19598 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.622745 19477 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.644753 19378 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.663399 19378 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.663915 19378 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.646735 19587 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:04:22.666114 19598 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.629570 19596 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.683928 19356 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.684211 19596 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.684546 19356 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:22.685952 19612 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:04:22.686944 19356 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:22.687198 19583 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.518s	user 0.000s	sys 0.001s
I20260709 13:04:22.687518 19356 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.688517 19513 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:22.689258 19454 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.212s	user 0.011s	sys 0.002s
I20260709 13:04:22.691030 19454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59138 (request call id 1118) took 4090 ms. Trace:
I20260709 13:04:22.662460 19526 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.691577 19454 rpcz_store.cc:276] 0709 13:04:18.759356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.760079 (+   723us) service_pool.cc:224] Handling call
0709 13:04:19.233905 (+473826us) ts_tablet_manager.cc:934] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
0709 13:04:21.095894 (+1861989us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:04:21.374878 (+278984us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:21.397987 (+ 23109us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:04:22.689917 (+1291930us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:04:22.690358 (+   441us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:22.690795 (+   437us) ts_tablet_manager.cc:989] Replica registered.
0709 13:04:22.690892 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":75585,"tablet-copy.queue_time_us":79577,"thread_start_us":458766,"threads_started":2}
I20260709 13:04:22.694690 19601 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:22.697259 19587 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.713593 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:22.710404 19515 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.001s
I20260709 13:04:22.720021 19596 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.721292 19600 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:22.722967 19382 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.724751 19477 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.708118 19526 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.718641 19587 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:22.729405 15842 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:22.712584 19601 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.753436 19600 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:22.757215 19600 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.758154 19601 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.749070 19596 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.759824 19587 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.761346 19596 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.761940 19526 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.762288 15840 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:22.730382 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:22.730953 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.701964 19501 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.733397 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.769891 15159 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:22.771703 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.734583 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.773269 15841 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:22.734858 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.736119 15233 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:22.732656 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.737032 19513 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.738896 19559 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.740371 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.741871 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.740552 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.688055 19356 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.744767 19563 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:22.732590 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.781590 15155 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:22.781589 15162 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:22.784735 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.785140 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.784758 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.786980 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.787708 15231 raft_consensus.cc:2410] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:22.790176 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.788218 15158 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:22.796017 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.771725 19501 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.750976 19515 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.781052 19371 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:22.805323 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.805847 15233 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:22.808233 15232 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:04:22.810477 19559 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.813131 19602 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.813675 19602 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:22.814404 19627 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:04:22.815896 19602 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.817458 19356 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.817566 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2209) took 1477 ms. Trace:
I20260709 13:04:22.794224 19497 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:04:22.818037 15160 rpcz_store.cc:276] 0709 13:04:21.339738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.339948 (+   210us) service_pool.cc:224] Handling call
0709 13:04:21.357285 (+ 17337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.559625 (+202340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.817283 (+1257658us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400287dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:22.817475 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464000384}
I20260709 13:04:22.821113 15839 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:22.821389 15233 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:22.821868 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.783738 19513 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.824241 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.824527 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:22.824978 15843 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:22.827173 19515 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:22.827895 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.833123 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.835626 15232 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:22.837045 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.837680 15162 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:22.837572 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.838356 15159 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:22.838632 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.839897 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.840777 19513 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.841177 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.841799 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.842363 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.844369 19454 tablet_copy_client.cc:806] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:22.845041 19454 tablet_copy_client.cc:670] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:22.845459 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:22.846105 15841 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 8.
I20260709 13:04:22.847218 15841 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:22.847666 15232 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:22.848297 15841 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:22.802459 19214 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:22.840562 15157 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:22.851186 15162 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:22.851558 15155 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:04:22.852347 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.853662 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:22.856339 19356 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:22.857226 19356 consensus_queue.cc:1243] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:22.859956 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:22.860534 19586 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.597s	user 0.005s	sys 0.000s
I20260709 13:04:22.861182 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.862241 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.863253 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.863711 15157 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:22.874310 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.874589 15159 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:22.875085 15155 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 7.
I20260709 13:04:22.875648 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.857331 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:22.878098 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 889) took 1426 ms. Trace:
I20260709 13:04:22.878511 15228 rpcz_store.cc:276] 0709 13:04:21.452016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.453223 (+  1207us) service_pool.cc:224] Handling call
0709 13:04:21.470348 (+ 17125us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:21.803687 (+333339us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:21.844724 (+ 41037us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.243568 (+398844us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:22.252332 (+  8764us) log.cc:844] Serialized 78 byte log entry
0709 13:04:22.252483 (+   151us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:22.252720 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.875232 (+622512us) spinlock_profiling.cc:243] Waited 9.66 ms on lock 0x7b1800451780. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:22.875426 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.875494 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.877866 (+  2372us) raft_consensus.cc:1685] finished
0709 13:04:22.877888 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.878043 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44313,"prepare.queue_time_us":63009,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":140,"raft.queue_time_us":134009,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":11026,"spinlock_wait_cycles":22223104,"thread_start_us":655822,"threads_started":2,"wal-append.queue_time_us":633849}
I20260709 13:04:22.856868 19600 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.889154 15159 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:22.915421 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:22.901237 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.918074 15162 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:22.918370 19580 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:04:22.919350 15162 pending_rounds.cc:85] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:22.919132 19378 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:22.919684 15162 pending_rounds.cc:107] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:22.919947 15162 raft_consensus.cc:2889] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:22.920511 19474 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:22.921582 19378 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:22.922477 15159 pending_rounds.cc:85] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:22.922825 15159 pending_rounds.cc:107] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:22.923127 15159 raft_consensus.cc:2889] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:22.925712 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 509162us
I20260709 13:04:22.928359 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:22.929210 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:22.929533 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.930390 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:22.917116 19600 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.931552 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.888059 19356 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:22.935622 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:22.936046 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.939404 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.940243 15232 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
W20260709 13:04:22.944101 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 163781us
I20260709 13:04:22.932020 19584 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:22.933267 19600 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.944880 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:22.948552 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:22.951592 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 703623us
I20260709 13:04:22.956737 19559 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:22.957176 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.958288 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.959239 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:22.959702 15232 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.961107 15157 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:22.966773 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.967480 15236 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:22.968286 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:22.971004 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:22.971316 15236 raft_consensus.cc:2410] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:04:22.973483 15232 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:22.974001 19580 consensus_queue.cc:1243] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:22.974457 19599 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:22.951472 19513 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:22.934602 19589 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:22.977735 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48586.9us
W20260709 13:04:22.975519 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.962074 19356 consensus_queue.cc:1243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:22.989071 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:22.986627 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:22.987386 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:22.992224 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47454.5us
I20260709 13:04:22.992607 15236 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:22.989809 19584 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.003950 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.006477 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:22.984635 19558 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:22.982820 19399 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:04:23.008574 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:23.010598 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.009016 19558 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:22.993588 19476 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:23.018508 19558 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:23.071830 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65449.5us
I20260709 13:04:23.018198 19513 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.080386 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.015635 19487 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:04:23.083016 19562 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.381s	user 0.000s	sys 0.004s
I20260709 13:04:23.016212 19580 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:04:23.018287 19524 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:23.088613 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 99611.3us
I20260709 13:04:23.017319 19420 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:23.109333 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 122771us
W20260709 13:04:23.118758 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 110277us
I20260709 13:04:23.119575 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.123899 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.080585 19535 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:23.090185 19399 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.079455 19476 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.142424 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:23.090729 19524 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:23.150662 19518 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:23.151269 19518 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:23.153481 19412 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:23.153998 19412 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:23.155090 19412 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.146090 19476 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.078058 19575 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:04:23.172492 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.156628 19412 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.079521 19635 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:04:23.131882 19592 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:23.132009 19553 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:23.122717 19487 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:23.159580 19474 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:23.148150 19559 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:23.159044 19524 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:23.171991 19599 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.152313 19518 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.205590 19518 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.144918 19471 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:04:23.227762 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 55346.5us
I20260709 13:04:23.228946 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:23.191231 19592 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.230109 19592 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.189072 19576 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:23.231276 19589 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:23.187261 19605 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.159428 19596 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:04:23.232527 19605 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.235239 19605 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.231773 19589 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.236536 19589 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.238131 19589 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.213519 19474 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.239450 19474 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.245000 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 890) took 1792 ms. Trace:
I20260709 13:04:23.245499 15234 rpcz_store.cc:276] 0709 13:04:21.452386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.453813 (+  1427us) service_pool.cc:224] Handling call
0709 13:04:22.852204 (+1398391us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:04:22.852404 (+   200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.972298 (+119894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.972332 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.972805 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.109537 (+136732us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:23.109577 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.109647 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.236874 (+127227us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b48002610d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:04:23.244768 (+  7894us) raft_consensus.cc:1685] finished
0709 13:04:23.244793 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.244940 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109049,"raft.queue_time_us":117616,"raft.run_cpu_time_us":2768,"raft.run_wall_time_us":263650,"spinlock_wait_cycles":45729408,"thread_start_us":136348,"threads_started":1,"wal-append.queue_time_us":268299}
I20260709 13:04:23.188546 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2211) took 1510 ms. Trace:
I20260709 13:04:23.247589 15163 rpcz_store.cc:276] 0709 13:04:21.678456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.679080 (+   624us) service_pool.cc:224] Handling call
0709 13:04:21.706160 (+ 27080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.706406 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:21.785335 (+ 78929us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.720912 (+935577us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b1800429d00. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:22.995585 (+274673us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b1800429d00. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:04:23.003640 (+  8055us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.186849 (+183209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.186918 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:23.187096 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.187185 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.188481 (+  1296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1159234,"prepare.queue_time_us":1300929,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":101,"raft.queue_time_us":5645,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":3535,"spinlock_wait_cycles":27384960,"thread_start_us":182726,"threads_started":1,"wal-append.queue_time_us":239354}
W20260709 13:04:23.251132 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.187530 19614 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:04:23.253785 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.148316 19588 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:04:23.258365 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.145360 19598 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:04:23.200265 19595 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:23.267445 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 143652us
I20260709 13:04:23.200191 19559 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:23.269131 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.171272 19476 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.264825 19553 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.283442 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.268291 19576 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:23.295308 19576 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.271837 19559 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.187546 19584 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.281785 19458 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.314021 19592 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.317291 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1098) took 1927 ms. Trace:
I20260709 13:04:23.317734 15839 rpcz_store.cc:276] 0709 13:04:21.389667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.390189 (+   522us) service_pool.cc:224] Handling call
0709 13:04:21.414023 (+ 23834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.821551 (+1407528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.317064 (+495513us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54004b55d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:23.317236 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3235575552}
I20260709 13:04:23.317313 19600 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.322440 19592 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:23.326110 19514 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.327843 19078 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.333518 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1097) took 1959 ms. Trace:
I20260709 13:04:23.334071 15847 rpcz_store.cc:276] 0709 13:04:21.374604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.374817 (+   213us) service_pool.cc:224] Handling call
0709 13:04:21.407680 (+ 32863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.607974 (+1200294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.333173 (+725199us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540083e5d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:23.333440 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742543744}
I20260709 13:04:23.213984 19513 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.305451 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.304795 19576 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:23.337302 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2183) took 1539 ms. Trace:
I20260709 13:04:23.340626 15229 rpcz_store.cc:276] 0709 13:04:21.798074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.067924 (+269850us) service_pool.cc:224] Handling call
0709 13:04:22.068409 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.088096 (+ 19687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.088147 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.862049 (+773902us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b2c00032e28. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:04:22.862453 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.979759 (+117306us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:23.293773 (+314014us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.298718 (+  4945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.298823 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.298877 (+    54us) raft_consensus.cc:1685] finished
0709 13:04:23.337041 (+ 38164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.337219 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23424,"prepare.queue_time_us":31596,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":134099,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":37013,"spinlock_wait_cycles":50395520,"thread_start_us":413155,"threads_started":2,"wal-append.queue_time_us":219789}
I20260709 13:04:23.344595 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 883) took 2159 ms. Trace:
I20260709 13:04:23.345070 15235 rpcz_store.cc:276] 0709 13:04:21.185399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.196432 (+ 11033us) service_pool.cc:224] Handling call
0709 13:04:21.196806 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.324025 (+127219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.324064 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.325670 (+  1606us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.599283 (+273613us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:21.631274 (+ 31991us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.677682 (+ 46408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.679356 (+  1674us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.300929 (+1621573us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b24000c92a8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c77d9c4 00007f0d3c77bacb 00007f0d3c936dc9 00007f0d3c936a55 00007f0d3c935fb0 00007f0d3f5ac488 00007f0d3f5acbe8 00007f0d3f584398 00007f0d3f57f76a 00007f0d3f57ebb6
0709 13:04:23.344015 (+ 43086us) raft_consensus.cc:1685] finished
0709 13:04:23.344118 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.344255 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20199,"prepare.queue_time_us":74508,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":7509,"raft.queue_time_us":218143,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":74360,"spinlock_wait_cycles":29619072,"thread_start_us":346848,"threads_started":2,"wal-append.queue_time_us":456517}
W20260709 13:04:23.349238 19636 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:23.301940 19563 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:23.335235 19634 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:04:23.315904 19558 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:23.366225 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60867.5us
I20260709 13:04:23.367511 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:23.369216 19588 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:23.369758 19588 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.370618 19588 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.371157 19595 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.383118 19559 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.387555 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.011096 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.398679 19558 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.402475 19452 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.405028 19412 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:23.406278 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2208) took 2072 ms. Trace:
I20260709 13:04:23.406944 19476 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:23.372267 19595 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.408587 19624 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.406693 15161 rpcz_store.cc:276] 0709 13:04:21.333504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.333780 (+   276us) service_pool.cc:224] Handling call
0709 13:04:21.340786 (+  7006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.495754 (+1154968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.406044 (+910290us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540013e7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:23.406219 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2654796160}
I20260709 13:04:23.409650 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.411330 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:23.418757 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.419318 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.419780 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.422180 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:23.423013 15086 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:23.364105 19614 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:23.429826 15084 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:04:23.430400 15163 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:04:23.431733 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:23.417063 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.432504 15087 raft_consensus.cc:2376] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8. Re-sending same reply.
I20260709 13:04:23.442585 19214 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.443239 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.443120 19497 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:23.444401 19356 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.445003 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.446149 19497 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:23.448521 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 718164us
W20260709 13:04:23.450763 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.417243 19412 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:23.455334 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:23.409317 19624 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.424410 19602 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:23.456569 15080 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.467698 15087 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:04:23.468766 15085 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:23.471693 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:23.475847 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:23.476003 15203 connection.cc:256] RPC call timeout handler was delayed by -1.5616968199999519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:23.479059 19614 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.479697 19614 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.481185 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.481887 19614 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.482357 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.483906 19614 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.483978 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:23.485524 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.494706 15236 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:23.495929 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.468936 19356 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:23.461680 19587 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:23.497543 15232 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:23.497812 15847 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:23.497917 15840 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:23.427384 19420 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.510373 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.513262 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.514021 15234 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:23.516398 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 236565us
I20260709 13:04:23.506474 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.517935 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:23.503317 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:23.508180 15230 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:23.501711 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:23.519811 15236 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.521355 15159 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:04:23.525106 15159 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:23.526466 15845 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:23.526679 15841 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:23.522317 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.485844 19472 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:23.529304 15848 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:23.513801 19382 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:23.537595 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.474469 19214 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:04:23.530082 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:23.544989 15234 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.540895 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:23.520753 19559 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.484788 19614 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.524334 19655 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:23.523953 19587 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:23.558297 15080 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:23.549818 19478 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:04:23.541452 19382 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:23.564400 19382 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:23.566398 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61692.3us
I20260709 13:04:23.567607 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:23.570853 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:23.574445 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:23.575178 15080 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:23.576431 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:23.577872 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.548549 19477 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:23.578790 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:23.587894 15236 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:23.530345 19596 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:04:23.602202 15233 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.604475 19563 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:23.569885 19514 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:23.543018 19558 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:04:23.574910 19599 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:04:23.532949 19598 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:04:23.612700 19478 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.621206 19598 consensus_queue.cc:1243] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:23.621876 19478 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.611845 15237 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:23.600649 19420 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.612468 19371 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:04:23.623256 15234 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:23.623523 19371 consensus_queue.cc:1243] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:23.624051 19371 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:23.624291 19478 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.624455 19572 raft_consensus.cc:2764] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:04:23.625149 19587 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.550186 19602 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:23.629364 19601 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:23.629835 19601 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:23.630774 19601 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.631600 19576 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.632294 19601 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.632690 15229 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:23.633256 19598 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.633755 19598 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.617244 19596 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:23.636628 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:23.638959 15843 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.610847 19477 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:23.624361 19371 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:23.645412 19477 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.646019 19371 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.602813 19503 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:23.650532 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.651435 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:23.652648 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.653362 15841 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:23.653164 19565 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:23.655339 15841 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:04:23.655614 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79293.2us
I20260709 13:04:23.656533 15841 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:23.657089 19524 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.661545 19624 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.613643 19595 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:23.662379 19382 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.664285 19598 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:23.741930 19382 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:23.647848 19371 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.603008 19559 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:23.669082 15846 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:04:23.744812 19559 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:23.745304 19559 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.746284 19559 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.747773 19559 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.650043 19503 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.750384 19503 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:23.670220 19578 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:23.753031 19578 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.754137 19578 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.755760 19578 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.638468 19596 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.671939 19564 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:23.757297 19564 consensus_queue.cc:1243] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:23.673614 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:23.673202 19655 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:23.757738 19596 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.758415 19655 consensus_queue.cc:1243] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:23.759071 19655 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:23.759305 19596 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.759471 19655 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.678061 19600 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:23.679759 15848 raft_consensus.cc:1217] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Deduplicated request from leader. Original: 8.9->[9.10-10.11]   Dedup: 10.11->[]
I20260709 13:04:23.761444 15848 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [9.10-10.11]
I20260709 13:04:23.684880 15843 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:23.697804 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.700990 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.707603 15233 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:23.764458 15234 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:23.727145 15229 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:23.764869 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.765612 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.760633 19655 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.727975 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:23.764866 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 91330.7us
I20260709 13:04:23.767489 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:23.768937 19563 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:23.769294 19563 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.769614 19563 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.769948 19572 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.770457 19572 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:23.739818 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.766803 19655 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.774000 19524 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:23.774502 15083 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:04:23.765030 19599 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:23.775380 19599 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:23.731352 19477 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.777076 15839 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:23.780519 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.805904 15236 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:23.807569 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.811123 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:23.811930 15159 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:23.827348 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.842406 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:23.842473 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 145109us
W20260709 13:04:23.851642 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.852660 19416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.018s
W20260709 13:04:23.853693 19416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.018s	sys 0.018s
I20260709 13:04:23.724949 19472 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.899011 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:23.772684 19572 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:23.904345 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:23.905308 15237 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:23.838798 19214 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:23.927129 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 380410us
W20260709 13:04:23.929235 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.930142 15234 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:23.931144 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:23.932096 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:23.937990 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1138) took 1436 ms (client timeout 1913 ms). Trace:
W20260709 13:04:23.938563 15846 rpcz_store.cc:269] 0709 13:04:22.487540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.487821 (+   281us) service_pool.cc:224] Handling call
0709 13:04:23.923731 (+1435910us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002a12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:23.923934 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2713654016}
I20260709 13:04:23.879732 19505 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.841569 19614 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:23.938488 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 96439.8us
I20260709 13:04:23.952123 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:23.953481 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.910272 19592 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.967780 19524 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:23.967841 19659 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:04:23.968636 19664 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:23.969763 19452 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:23.969934 19553 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:23.970333 19452 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.970412 19553 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.971432 19452 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.973567 15846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:23.957497 19378 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:23.974828 19378 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.975210 19378 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.977649 15157 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:04:23.978556 19600 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:23.969796 19664 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:23.979075 19600 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:23.979344 19664 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:23.979535 19654 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:23.980063 19600 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.934569 19214 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:23.983669 14985 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 261) took 44886 ms (client timeout 15000 ms). Trace:
I20260709 13:04:23.986579 19602 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.971428 19553 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.966055 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:23.981374 19592 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.982227 19356 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:23.958580 19505 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.912679 19472 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.980252 19664 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:23.987136 19371 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:24.028443 19664 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:24.010394 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.032650 19553 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:24.033586 19668 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:04:24.034142 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:24.035053 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:24.035727 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:24.036568 15156 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:24.035781 15154 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:24.038822 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:24.039374 15154 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:24.040442 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:24.040881 15154 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:24.041320 19598 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:24.041713 19598 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.042047 19598 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:24.042518 15154 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:04:24.016497 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 407031us
I20260709 13:04:24.039759 19592 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:24.045506 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.045645 19356 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.047938 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.052525 19599 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.027098 15159 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:24.055914 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:24.061873 19669 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:24.062381 19237 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:24.062481 19669 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.064920 19669 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:24.065603 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.066919 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.067538 15083 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:24.068971 15083 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:04:24.069244 19237 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:24.008973 14985 rpcz_store.cc:269] 0709 13:03:39.104214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.139173 (+ 34959us) service_pool.cc:224] Handling call
0709 13:04:23.983588 (+44844415us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:20.496447 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:20.497394 (+   947us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:20.497448 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:04:20.497491 (+    43us) tablet.cc:662] Decoding operations
0709 13:04:20.498333 (+   842us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:20.498451 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:20.498485 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:04:20.499237 (+   752us) tablet.cc:701] Row locks acquired
0709 13:04:20.499295 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:04:20.499502 (+   207us) write_op.cc:270] Start()
0709 13:04:20.499665 (+   163us) write_op.cc:276] Timestamp: P: 1783602260499426 usec, L: 0
0709 13:04:20.499715 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:04:20.500475 (+   760us) log.cc:844] Serialized 2084 byte log entry
0709 13:04:20.789049 (+288574us) op_driver.cc:464] REPLICATION: finished
0709 13:04:21.033330 (+244281us) write_op.cc:301] APPLY: starting
0709 13:04:21.033424 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:21.830873 (+797449us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:21.884753 (+ 53880us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.237906 (+353153us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.351180 (+113274us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:04:22.372625 (+ 21445us) write_op.cc:312] APPLY: finished
0709 13:04:22.503828 (+131203us) log.cc:844] Serialized 57 byte log entry
0709 13:04:22.631001 (+127173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.634803 (+  3802us) write_op.cc:454] Released schema lock
0709 13:04:22.646873 (+ 12070us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.666118 (+ 19245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.666157 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":894453,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":243967,"apply.run_cpu_time_us":88545,"apply.run_wall_time_us":1627519,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25057,"num_ops":5,"prepare.queue_time_us":26803,"prepare.run_cpu_time_us":22762,"prepare.run_wall_time_us":74164,"raft.queue_time_us":91958,"raft.run_cpu_time_us":18104,"raft.run_wall_time_us":152854,"replication_time_us":281144,"thread_start_us":313973,"threads_started":5,"wal-append.queue_time_us":276511,"wal-append.run_cpu_time_us":28021,"wal-append.run_wall_time_us":2120137}]]}
I20260709 13:04:24.070657 19237 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.073122 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 107151us
I20260709 13:04:24.074425 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:24.075204 19371 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:24.075673 19371 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:24.077708 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:24.079367 15083 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:24.136022 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:24.136613 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.137146 15839 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:24.138298 15079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.010s
W20260709 13:04:24.138722 15079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.007s	sys 0.010s
I20260709 13:04:24.010946 19524 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:24.019069 19670 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:24.151983 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:24.138068 19578 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.145044 19078 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:24.044634 19563 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:24.168553 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.018919 19602 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:24.171519 15052 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 -> Peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Couldn't send request to peer 3c28be6f86ea4fb6b79a47d3999f5a82. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:24.200519 15160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.024s	sys 0.074s
I20260709 13:04:24.140954 19584 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:04:24.196391 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.076535 19371 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:24.171085 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93480.9us
I20260709 13:04:24.202417 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.200757 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:24.205963 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:24.164873 19382 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:24.213085 19564 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:24.213675 19564 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.214097 19564 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:24.214798 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.002s	sys 0.023s
W20260709 13:04:24.215482 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.011s	sys 0.015s
W20260709 13:04:24.215853 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.011s	sys 0.015s
I20260709 13:04:24.217061 19595 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:24.217401 19412 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:24.219200 19412 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:24.200912 15160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.025s	sys 0.074s
W20260709 13:04:24.215229 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.002s	sys 0.023s
W20260709 13:04:24.225656 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 755908us
I20260709 13:04:24.229776 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:24.238402 19588 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.241966 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1075) took 2288 ms. Trace:
I20260709 13:04:24.242323 19588 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.242498 15088 rpcz_store.cc:276] 0709 13:04:21.954014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.431586 (+477572us) service_pool.cc:224] Handling call
0709 13:04:22.481301 (+ 49715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.483833 (+  2532us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:22.650762 (+166929us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.708465 (+ 57703us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.407759 (+699294us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b18003f3760. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:24.241340 (+833581us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b18003f3760. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287
0709 13:04:24.241475 (+   135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:24.241546 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:24.241614 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.241707 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.241888 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":667973,"prepare.queue_time_us":178720,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":98,"raft.queue_time_us":701576,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":29901,"spinlock_wait_cycles":71186816,"thread_start_us":640883,"threads_started":3,"wal-append.queue_time_us":135776,"wal-append.run_cpu_time_us":2078,"wal-append.run_wall_time_us":1047439}
I20260709 13:04:24.239070 19595 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.260327 19595 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.261031 19572 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.262753 19595 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.261773 19572 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.264778 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.265568 15844 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:24.266978 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.267710 15084 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:24.269348 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.269523 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:24.245507 19588 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:24.257441 19078 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:24.250083 15839 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:24.259562 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.270645 19595 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:24.273206 19595 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.273856 19078 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:24.274204 19595 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:24.270109 14981 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.290644 15087 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:24.284766 19572 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:24.292824 19592 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.293836 19592 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.161000 19356 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.261336 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:24.286578 14981 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 8, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.310192 15080 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:24.317373 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.296983 15082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.032s	sys 0.014s
I20260709 13:04:24.295912 15083 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
W20260709 13:04:24.206081 19526 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.046s	sys 0.088s
W20260709 13:04:24.299715 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77155.4us
W20260709 13:04:24.320909 15082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.033s	sys 0.014s
I20260709 13:04:24.323328 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:24.308449 19573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.212s	user 0.010s	sys 0.045s
I20260709 13:04:24.173718 19578 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.294878 19669 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:24.352998 15159 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:24.360585 15236 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:24.353562 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:24.297464 19584 consensus_queue.cc:1243] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:24.299594 19670 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:24.367957 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:24.166003 19214 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.298296 19382 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:24.399353 19382 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.344946 19526 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.046s	sys 0.088s
I20260709 13:04:24.202201 19503 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.311932 19477 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.382869 19674 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:24.386999 19356 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:24.345366 19588 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:04:24.389695 19378 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.391458 19669 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.369042 19576 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.393810 19602 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.394308 19563 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:24.395750 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:24.356771 19573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.269s	user 0.010s	sys 0.046s
I20260709 13:04:24.413130 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:24.413552 19559 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.003s
I20260709 13:04:24.414458 19564 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.415850 19663 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:24.305622 19572 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:04:24.422859 15237 pending_rounds.cc:85] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 7
W20260709 13:04:24.384635 19596 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.001s
W20260709 13:04:24.374603 19666 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:04:24.427608 14981 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:24.351504 19624 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:04:24.357803 19078 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:24.360357 19524 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.434307 19514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.010s	sys 0.014s
I20260709 13:04:24.467159 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2718) took 1062 ms. Trace:
W20260709 13:04:24.544495 15086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.031s	sys 0.031s
W20260709 13:04:24.551795 15086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.032s	sys 0.031s
I20260709 13:04:24.553315 15232 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:24.545006 15237 pending_rounds.cc:107] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:24.544931 15156 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:24.555193 15237 raft_consensus.cc:2889] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.437140 19670 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:24.429157 19592 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.439577 19214 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:24.511092 19655 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.000s
I20260709 13:04:24.560578 14981 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 9, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:24.463791 19669 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.446182 19588 consensus_queue.cc:1243] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:24.552996 19598 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
W20260709 13:04:24.581251 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.028s	sys 0.035s
W20260709 13:04:24.593055 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.028s	sys 0.035s
W20260709 13:04:24.594051 15162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.003s	sys 0.031s
W20260709 13:04:24.597605 15162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.003s	sys 0.031s
I20260709 13:04:24.586558 14981 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.518679 19505 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.599327 19503 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.600080 19600 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:24.601285 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2631) took 1178 ms. Trace:
I20260709 13:04:24.638659 15082 rpcz_store.cc:276] 0709 13:04:23.423363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.423469 (+   106us) service_pool.cc:224] Handling call
0709 13:04:23.423766 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.459033 (+ 35267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.459085 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.459770 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.338667 (+878897us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:24.338708 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.338777 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.601051 (+262274us) raft_consensus.cc:1685] finished
0709 13:04:24.601075 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.601223 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25767,"raft.queue_time_us":26011,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9593,"thread_start_us":878452,"threads_started":1,"wal-append.queue_time_us":1139268}
I20260709 13:04:24.646837 15228 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:24.647351 15228 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:24.655874 19662 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:24.674480 19420 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:24.562598 19670 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.679527 19670 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.694428 19078 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.553491 19663 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.699460 19663 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.701359 19378 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:24.701996 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.704991 19578 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.596063 19598 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.708168 19598 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:24.710063 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:24.711068 15843 raft_consensus.cc:2368] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:24.713160 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.714818 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:24.717715 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:24.720176 15235 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.009s	sys 0.028s
W20260709 13:04:24.720664 15235 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.009s	sys 0.029s
W20260709 13:04:24.724711 15848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.015s	sys 0.023s
I20260709 13:04:24.559744 19514 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.603598 19614 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:24.607758 19686 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:24.729657 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:24.730358 15159 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:24.559415 19559 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.610114 19472 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:24.594897 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:24.558280 19601 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:24.553289 19624 consensus_queue.cc:1243] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:24.561681 19563 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:24.551064 15158 rpcz_store.cc:276] 0709 13:04:23.409001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.412052 (+  3051us) service_pool.cc:224] Handling call
0709 13:04:23.428682 (+ 16630us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:23.437918 (+  9236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.437962 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:23.448262 (+ 10300us) log.cc:844] Serialized 52 byte log entry
0709 13:04:23.940262 (+492000us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:24.007316 (+ 67054us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.286656 (+279340us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b18000eb840. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:04:24.288611 (+  1955us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.288869 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.288966 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.466875 (+177909us) raft_consensus.cc:1685] finished
0709 13:04:24.466905 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.467085 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":237792,"prepare.queue_time_us":346385,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":93,"raft.queue_time_us":271710,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":28170,"spinlock_wait_cycles":13484928,"thread_start_us":441947,"threads_started":1,"wal-append.queue_time_us":878174}
I20260709 13:04:24.636276 14981 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 7, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.520006 19678 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 13:04:24.726984 15848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.015s	sys 0.023s
I20260709 13:04:24.732355 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 910) took 1250 ms. Trace:
W20260709 13:04:24.734931 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 140160us
I20260709 13:04:24.737155 15052 leader_election.cc:400] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:24.648212 15228 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.739606 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:24.740886 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:24.726545 19599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.022s	sys 0.018s
I20260709 13:04:24.728291 19614 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:24.724756 19588 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:04:24.726143 19655 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:04:24.753790 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.780687 15082 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:24.754045 15228 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:24.756160 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:24.783746 15083 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:24.757063 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:24.757742 14981 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.786855 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:24.761608 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.008s	sys 0.041s
W20260709 13:04:24.788128 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.008s	sys 0.041s
W20260709 13:04:24.788573 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 731206us
I20260709 13:04:24.753309 15235 rpcz_store.cc:276] 0709 13:04:23.479568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.479696 (+   128us) service_pool.cc:224] Handling call
0709 13:04:23.480101 (+   405us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:23.480495 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.480525 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:23.481196 (+   671us) log.cc:844] Serialized 78 byte log entry
0709 13:04:23.940702 (+459506us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:24.721217 (+780515us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.729745 (+  8528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.729820 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.729865 (+    45us) raft_consensus.cc:1685] finished
0709 13:04:24.729878 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.730043 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13401,"raft.queue_time_us":10651,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7913,"thread_start_us":1173480,"threads_started":2,"wal-append.queue_time_us":834697}
I20260709 13:04:24.754716 15839 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:24.794590 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:24.779874 19595 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.504s	user 0.013s	sys 0.000s
I20260709 13:04:24.797106 19670 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:24.807585 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.027s	sys 0.047s
W20260709 13:04:24.747324 19416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.000s	sys 0.060s
I20260709 13:04:24.791167 19078 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.762295 19674 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:24.734278 19514 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.757673 19599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.022s	sys 0.018s
I20260709 13:04:24.801296 19588 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:24.794581 19655 consensus_queue.cc:1243] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:04:24.801064 19518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.028s	sys 0.029s
W20260709 13:04:24.808001 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.027s	sys 0.047s
I20260709 13:04:24.754247 19563 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:24.749493 19559 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.803464 19587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s	user 0.027s	sys 0.039s
W20260709 13:04:24.809921 19416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.000s	sys 0.060s
I20260709 13:04:24.808729 19670 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:24.829792 19670 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:24.808804 19595 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.823980 19518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.028s	sys 0.029s
I20260709 13:04:24.838297 15161 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:24.842737 15158 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:24.856377 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:24.844705 19514 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:24.846539 19563 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:24.855611 19548 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.013s	sys 0.019s
I20260709 13:04:24.869038 15231 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:24.816252 19078 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:24.874568 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.001s	sys 0.023s
W20260709 13:04:24.875147 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.001s	sys 0.023s
W20260709 13:04:24.875646 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 535976us
I20260709 13:04:24.876899 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:24.881714 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.008s	sys 0.018s
W20260709 13:04:24.882273 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.008s	sys 0.018s
W20260709 13:04:24.883034 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 622650us
I20260709 13:04:24.747052 15159 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:24.816471 19588 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:24.806582 14981 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 8, now has a pending config: VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193), VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194), VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195), VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:24.811578 19686 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
W20260709 13:04:24.834103 19587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.027s	sys 0.039s
I20260709 13:04:24.833410 19674 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:24.860769 15228 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:04:24.889420 15085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.029s	sys 0.029s
W20260709 13:04:24.910122 15085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.029s	sys 0.029s
W20260709 13:04:24.872126 19548 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.014s	sys 0.020s
I20260709 13:04:24.907311 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 898) took 1147 ms. Trace:
I20260709 13:04:24.924053 15161 rpcz_store.cc:276] 0709 13:04:23.760221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.789434 (+ 29213us) service_pool.cc:224] Handling call
0709 13:04:23.897872 (+108438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.854909 (+957037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.907000 (+ 52091us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b540033f754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:24.907232 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161811200}
W20260709 13:04:24.889863 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.020s	sys 0.038s
W20260709 13:04:24.925450 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.021s	sys 0.038s
I20260709 13:04:24.911677 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:24.872563 19602 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:24.928213 14981 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 8, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:04:24.932638 15229 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.021s	sys 0.000s
W20260709 13:04:24.935365 15229 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.021s	sys 0.000s
I20260709 13:04:24.938892 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:24.874653 19513 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:04:24.928452 19576 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 13:04:24.936203 19237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.013s
I20260709 13:04:24.959123 14981 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:24.965691 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54102.9us
I20260709 13:04:24.966111 15082 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:24.928208 19669 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:24.968647 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:24.969835 15080 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:24.972023 19237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.008s	sys 0.014s
W20260709 13:04:25.022749 19691 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:04:24.997473 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.029302 15082 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:25.000962 15235 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:25.002156 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.031028 15232 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:25.005085 15230 pending_rounds.cc:85] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 9
I20260709 13:04:25.032073 15230 pending_rounds.cc:107] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:25.032370 15230 raft_consensus.cc:2889] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:25.016307 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.034209 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:25.020903 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.038975 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.039644 15232 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:25.005658 19614 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:25.041944 19614 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:25.006183 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.045684 15234 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:24.930835 19214 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:25.046780 19214 consensus_queue.cc:1243] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:24.998875 15231 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [10.10-11.11]
I20260709 13:04:25.020318 14981 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 9, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 9 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:25.008118 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.053048 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.053956 15082 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:25.054240 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:25.017877 15163 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:25.055269 15163 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:25.020226 15155 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
W20260709 13:04:25.058229 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50184.6us
I20260709 13:04:24.987574 19697 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
W20260709 13:04:24.987983 19681 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:25.061443 15082 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:25.062521 15158 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 13:04:25.064860 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.065241 15080 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:25.065441 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:24.995975 19477 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.987733 19602 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:24.990607 19624 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.067003 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:25.057080 19665 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:04:25.016644 19688 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.059s	user 0.000s	sys 0.003s
I20260709 13:04:25.018576 19416 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.071022 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.080145 15084 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:25.071576 19503 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:25.034585 19572 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:25.059481 15163 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:25.074920 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.076707 14981 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 10, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 10 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.081782 15163 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:25.066495 19452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.009s	sys 0.033s
I20260709 13:04:25.053824 19659 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:25.082015 14981 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 9, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:25.073944 15082 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:25.063130 19564 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:25.007159 19678 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:04:25.104076 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.106725 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2632) took 1680 ms. Trace:
I20260709 13:04:25.106946 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 894) took 1432 ms. Trace:
W20260709 13:04:25.108948 15200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.005s	sys 0.015s
W20260709 13:04:25.110280 15200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.005s	sys 0.015s
W20260709 13:04:25.112033 15203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.742s (SENT) [suppressed 18 similar messages]
I20260709 13:04:25.061072 19664 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:04:25.107175 15086 rpcz_store.cc:276] 0709 13:04:23.426181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.426353 (+   172us) service_pool.cc:224] Handling call
0709 13:04:23.426669 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.450037 (+ 23368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.450082 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.842028 (+391946us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b2c000ecb98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:04:23.842433 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.552516 (+710083us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:24.552577 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.561771 (+  9194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.016026 (+454255us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:25.106527 (+ 90501us) raft_consensus.cc:1685] finished
0709 13:04:25.106549 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.106670 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20485,"raft.queue_time_us":32581,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6275,"spinlock_wait_cycles":121002752,"thread_start_us":649021,"threads_started":1,"wal-append.queue_time_us":970870}
I20260709 13:04:25.119055 15082 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:04:25.086779 19452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.009s	sys 0.033s
I20260709 13:04:25.119853 14981 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.123924 15082 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:04:25.126308 19078 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.065208 19669 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:04:25.084234 19602 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:25.084084 19572 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:25.127424 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 901) took 1142 ms. Trace:
I20260709 13:04:25.085143 19559 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.128283 15158 raft_consensus.cc:1217] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.8->[7.9-7.9]   Dedup: 7.9->[]
I20260709 13:04:25.138993 15158 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:04:25.115119 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.132562 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:25.144735 15086 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.075877 19477 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.107483 15162 rpcz_store.cc:276] 0709 13:04:23.674798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.674968 (+   170us) service_pool.cc:224] Handling call
0709 13:04:23.675310 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:23.842229 (+166919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.842259 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:23.926424 (+ 84165us) log.cc:844] Serialized 52 byte log entry
0709 13:04:24.612438 (+686014us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:24.612484 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.614765 (+  2281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.022197 (+407432us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:25.087257 (+ 65060us) raft_consensus.cc:1685] finished
0709 13:04:25.087288 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.106888 (+ 19600us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113799,"raft.queue_time_us":214401,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":11323,"spinlock_wait_cycles":115071360,"thread_start_us":674385,"threads_started":1,"wal-append.queue_time_us":943469}
I20260709 13:04:25.134979 15157 rpcz_store.cc:276] 0709 13:04:23.985053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.985173 (+   120us) service_pool.cc:224] Handling call
0709 13:04:23.985564 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:23.985928 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.985956 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:23.986603 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:04:24.821484 (+834881us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:24.821541 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.821631 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.127192 (+305561us) raft_consensus.cc:1685] finished
0709 13:04:25.127213 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.127357 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166224,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7151,"thread_start_us":833735,"threads_started":1,"wal-append.queue_time_us":1074323}
I20260709 13:04:25.137015 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.153923 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:25.112624 19526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.010s	sys 0.029s
I20260709 13:04:25.021839 19588 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:25.156483 15086 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:25.103080 19420 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:25.107910 19659 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:25.171744 15155 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
W20260709 13:04:25.157034 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.007s	sys 0.016s
W20260709 13:04:25.157691 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.173652 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.008s	sys 0.017s
I20260709 13:04:25.134322 19598 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.086730 19416 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:25.158687 15237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.008s	sys 0.015s
I20260709 13:04:25.158780 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.161849 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:25.186408 15237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.015s
I20260709 13:04:25.187237 15162 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:04:25.174093 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 833752us
W20260709 13:04:25.149592 19497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.020s	sys 0.022s
I20260709 13:04:25.188369 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:25.174465 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.157848 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:25.157166 19526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.011s	sys 0.029s
I20260709 13:04:25.189512 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.201763 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:25.202662 15082 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:25.150776 19595 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.196094 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.205857 15232 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
W20260709 13:04:25.153822 19382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.015s	sys 0.011s
W20260709 13:04:25.162115 19356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.027s	sys 0.016s
W20260709 13:04:25.195843 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 447017us
I20260709 13:04:25.204981 15088 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:25.211180 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.212041 15155 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:25.225955 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.226078 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.227063 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:25.203066 19497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.021s	sys 0.022s
I20260709 13:04:25.229274 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.178805 19564 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:25.245669 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.246708 15088 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:25.149072 19664 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:04:25.210768 19382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.016s	sys 0.011s
I20260709 13:04:25.206718 19588 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:25.200368 19599 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:25.210712 19356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.027s	sys 0.016s
I20260709 13:04:25.249814 14981 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.279852 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.249763 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:25.251102 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:25.254622 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:25.257424 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:25.282437 15842 raft_consensus.cc:2410] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:04:25.258208 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:25.258471 15841 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:25.258927 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:25.283586 15845 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.254515 19548 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:25.262547 19698 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:04:25.263435 19692 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:04:25.263543 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.212751 19596 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.296804 14981 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 2 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.271191 15081 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:25.272984 19663 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.188983 19576 consensus_queue.cc:1175] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 9 status { last_received { term: 9 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 9 index: 7 } }
I20260709 13:04:25.281538 15082 raft_consensus.cc:2376] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12. Re-sending same reply.
W20260709 13:04:25.315371 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.268390 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.341233 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.342180 19416 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:25.342424 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.343441 15157 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:04:25.344347 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.301818 15087 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:25.345479 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:25.346297 19601 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.001s
I20260709 13:04:25.346845 19601 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:25.346902 15157 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:25.291326 19602 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:25.357417 15080 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:04:25.357858 15080 pending_rounds.cc:85] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 9
I20260709 13:04:25.358073 15080 pending_rounds.cc:107] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:25.358299 15080 raft_consensus.cc:2889] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:25.359349 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:25.360450 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.201996 19598 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.225411 19655 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:04:25.361694 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:25.346174 15087 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:04:25.289913 19573 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:25.369443 19573 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:25.371106 19513 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:25.371665 19513 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.371708 19573 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:25.372228 19513 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:25.209892 19670 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.374624 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.374853 19670 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.179535 19420 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.377493 19420 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:25.337270 19596 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.380045 19596 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:25.336536 19659 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.383502 19513 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.384197 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.385411 19599 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:25.385892 19599 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.388301 19655 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:25.388737 19655 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:25.295027 19548 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.389819 15810 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:25.365767 19477 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.214242 19572 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.391116 19572 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:25.391472 19595 consensus_queue.cc:1175] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 8 status { last_received { term: 7 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:25.389667 19655 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.395040 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:25.395167 19655 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:25.362629 19598 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:25.395838 15157 raft_consensus.cc:2410] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:04:25.397140 15157 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:25.397238 19708 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:25.398630 19708 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:25.399137 19708 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:25.292016 15155 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:25.363189 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.403136 14981 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 10, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:25.405284 15080 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:25.406790 15080 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:25.410543 15843 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:25.426113 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.391252 15081 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:25.400082 19708 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.430353 15845 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:25.432119 15163 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:04:25.438932 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.020s	sys 0.026s
W20260709 13:04:25.446401 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.020s	sys 0.026s
W20260709 13:04:25.447124 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 660346us
I20260709 13:04:25.447422 15162 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:25.448546 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:25.450153 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:25.451560 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 198679us
I20260709 13:04:25.391037 19548 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:25.454702 14981 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 9, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:25.485948 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 889) took 1960 ms. Trace:
W20260709 13:04:25.461870 15079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.018s	sys 0.030s
I20260709 13:04:25.483693 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.485425 14981 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.486738 15163 rpcz_store.cc:276] 0709 13:04:23.525464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.525641 (+   177us) service_pool.cc:224] Handling call
0709 13:04:23.536520 (+ 10879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.017781 (+1481261us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b540015fa20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:04:25.445989 (+428208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.485591 (+ 39602us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540033fc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:25.485865 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3189979904}
I20260709 13:04:25.488085 14981 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 5 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.489593 15079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.018s	sys 0.030s
I20260709 13:04:25.454051 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:25.489626 19382 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.490691 19587 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:25.491011 19382 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:25.491300 19587 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.493014 19703 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.493918 19371 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.494479 19371 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.493577 19703 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.496026 19595 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.496656 19595 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.497058 19600 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:25.497707 19600 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:25.498164 19600 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:25.498503 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:25.499265 19595 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.501631 19382 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:25.519816 19624 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:25.526571 15084 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:04:25.526616 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.019s	sys 0.033s
I20260709 13:04:25.425437 19559 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.527724 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.019s	sys 0.033s
W20260709 13:04:25.453917 19698 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:04:25.531605 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77644.6us
I20260709 13:04:25.449416 19598 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:25.546850 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.547776 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:25.548460 15082 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:25.453701 19664 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:25.295312 19564 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.484308 19686 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.485934 19471 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.535719 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 745632us
W20260709 13:04:25.564728 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.580264 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 92598.4us
I20260709 13:04:25.582120 15128 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.583724 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:25.584939 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.605338 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.534864 19584 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:25.452808 19663 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.615106 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:25.594825 15843 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:25.598387 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.617712 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.600293 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.618949 15155 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:25.600637 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:25.622033 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:25.625255 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.584649 15236 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.015s
I20260709 13:04:25.541924 19518 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:25.531158 19588 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.544652 19356 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:25.638106 15843 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:04:25.577569 19524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.014s	sys 0.011s
W20260709 13:04:25.565804 19476 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.017s	sys 0.008s
I20260709 13:04:25.521905 19573 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:04:25.551079 19598 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:25.629531 15236 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.011s	sys 0.015s
I20260709 13:04:25.635453 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:25.667585 15839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.017s	sys 0.021s
I20260709 13:04:25.489250 19702 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
W20260709 13:04:25.662992 15161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.016s	sys 0.021s
W20260709 13:04:25.657042 15156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.009s	sys 0.021s
W20260709 13:04:25.657248 15229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.013s	sys 0.019s
I20260709 13:04:25.588979 19592 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:25.655689 15085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.014s	sys 0.015s
W20260709 13:04:25.671298 15085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.015s	sys 0.015s
W20260709 13:04:25.671868 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.015s	sys 0.025s
W20260709 13:04:25.672356 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.015s	sys 0.025s
I20260709 13:04:25.587628 19412 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:25.668466 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:25.669751 15229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.014s	sys 0.019s
I20260709 13:04:25.675371 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 920) took 1353 ms. Trace:
W20260709 13:04:25.669232 15161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.017s	sys 0.021s
W20260709 13:04:25.667946 15839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.017s	sys 0.021s
I20260709 13:04:25.551460 19559 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:25.665695 19476 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.019s	sys 0.009s
W20260709 13:04:25.660862 19578 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.017s	sys 0.014s
W20260709 13:04:25.669216 15156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.010s	sys 0.021s
I20260709 13:04:25.680449 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.587819 19471 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.645973 19356 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:25.679057 19524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.015s	sys 0.012s
I20260709 13:04:25.694018 15157 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:25.666633 19614 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.679711 19592 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.682705 19412 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.668443 19588 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.671193 19598 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:25.711511 19471 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:25.676558 19595 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:04:25.713176 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.718521 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.796221 15157 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:25.796456 15230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.005s	sys 0.008s
W20260709 13:04:25.796816 15230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.005s	sys 0.008s
W20260709 13:04:25.689611 19578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.018s	sys 0.014s
I20260709 13:04:25.675881 15229 rpcz_store.cc:276] 0709 13:04:24.321770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.323949 (+  2179us) service_pool.cc:224] Handling call
0709 13:04:24.324273 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.324540 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:24.935923 (+611383us) log.cc:844] Serialized 14 byte log entry
0709 13:04:25.674930 (+739007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:25.675004 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:25.675075 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.675166 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.675304 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":693369,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"thread_start_us":1313325,"threads_started":2}
W20260709 13:04:25.743521 15814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.008s	sys 0.022s
W20260709 13:04:25.749029 15154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.015s	sys 0.022s
W20260709 13:04:25.797537 15814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.023s
W20260709 13:04:25.797720 15154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.015s	sys 0.022s
I20260709 13:04:25.727643 15846 raft_consensus.cc:606] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:25.693919 19671 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:25.749466 15155 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:25.788136 15163 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:25.715051 19356 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.698387 19704 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.698150 19697 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:25.751560 15086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.000s	sys 0.011s
I20260709 13:04:25.802168 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:25.785450 19497 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:25.876019 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 912) took 1159 ms. Trace:
I20260709 13:04:25.891057 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:25.891956 15229 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:25.893052 15235 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:25.811683 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:25.836388 15203 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.005s	sys 0.029s
W20260709 13:04:25.894908 15203 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.005s	sys 0.034s
W20260709 13:04:25.797958 19563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.012s	sys 0.003s
W20260709 13:04:25.819331 15228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.006s	sys 0.009s
I20260709 13:04:25.817761 19524 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:25.898824 15086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.001s	sys 0.013s
W20260709 13:04:25.864638 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.900375 19564 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.808117 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:25.897428 15228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.012s	sys 0.011s
I20260709 13:04:25.723234 19614 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:25.849977 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:25.810796 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.010s	sys 0.005s
I20260709 13:04:25.907382 15847 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:25.908087 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.011s	sys 0.005s
W20260709 13:04:25.906275 15848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.048s
W20260709 13:04:25.911419 15848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.001s	sys 0.048s
I20260709 13:04:25.847728 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:25.909722 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:25.895552 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 783459us
I20260709 13:04:25.909440 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:25.899418 19524 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.789214 19513 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:25.807883 19452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.003s	sys 0.011s
I20260709 13:04:25.918855 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:25.988981 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.010s	sys 0.011s
W20260709 13:04:25.993486 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.010s	sys 0.011s
I20260709 13:04:25.912490 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1150) took 1863 ms. Trace:
W20260709 13:04:25.897039 19563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.013s	sys 0.003s
I20260709 13:04:25.896056 19592 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:25.873672 19714 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:04:25.878309 19713 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:25.962741 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.979346 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:26.016968 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:25.977824 19671 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.963879 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2642) took 1706 ms. Trace:
W20260709 13:04:25.877465 19710 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:04:25.809742 19686 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:25.895175 15157 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:25.928463 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:25.878995 19717 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:25.999939 15229 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:25.891556 15156 rpcz_store.cc:276] 0709 13:04:24.716811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.718564 (+  1753us) service_pool.cc:224] Handling call
0709 13:04:24.727466 (+  8902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.729684 (+  2218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.729722 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.735200 (+  5478us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.987174 (+251974us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18003ae2e0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:25.690194 (+703020us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:25.690256 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.690465 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.875682 (+185217us) raft_consensus.cc:1685] finished
0709 13:04:25.875700 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.875935 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":19986,"spinlock_wait_cycles":7988608,"thread_start_us":702763,"threads_started":1,"wal-append.queue_time_us":1110351}
I20260709 13:04:26.005101 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.006659 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:35386 (request call id 4) took 2030 ms. Trace:
I20260709 13:04:26.006709 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.007017 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.028780 15846 rpcz_store.cc:276] 0709 13:04:23.972467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.973427 (+   960us) service_pool.cc:224] Handling call
0709 13:04:26.003086 (+2029659us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b540049c0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:26.003213 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4032583936}
I20260709 13:04:26.007071 15087 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:25.929540 19477 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:25.973945 19371 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.884407 19588 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:25.892776 19655 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:26.017472 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.041102 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.916203 19237 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:26.022032 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.011s	sys 0.011s
I20260709 13:04:26.014189 19471 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:26.025419 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 97065.3us
I20260709 13:04:26.047546 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:25.862879 19596 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:04:26.019352 15085 rpcz_store.cc:276] 0709 13:04:24.257930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.258058 (+   128us) service_pool.cc:224] Handling call
0709 13:04:24.258399 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.910688 (+652289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.910734 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.926325 (+ 15591us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.671785 (+745460us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:25.671837 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.671930 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.963690 (+291760us) raft_consensus.cc:1685] finished
0709 13:04:25.963713 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.963825 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49253,"raft.queue_time_us":185819,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":63179,"thread_start_us":1347539,"threads_started":2,"wal-append.queue_time_us":977860}
I20260709 13:04:26.033239 15235 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:26.051754 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:25.819936 19600 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:25.995313 15848 rpcz_store.cc:276] 0709 13:04:24.049536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.049657 (+   121us) service_pool.cc:224] Handling call
0709 13:04:24.049903 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.050094 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:24.751337 (+701243us) log.cc:844] Serialized 14 byte log entry
0709 13:04:25.912001 (+1160664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:25.912077 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:25.912145 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.912241 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.912418 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16256,"prepare.queue_time_us":907174,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":59628,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":1842471,"threads_started":3,"wal-append.queue_time_us":947203}
W20260709 13:04:26.007880 19602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.015s	sys 0.018s
I20260709 13:04:26.009724 19703 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:26.042697 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.011s	sys 0.011s
W20260709 13:04:26.057283 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 558817us
I20260709 13:04:26.022547 19663 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:26.037978 19514 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:26.034720 19572 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:26.039566 19477 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.055684 19588 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:26.057395 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:25.930610 19704 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:26.052402 15156 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:26.194114 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136790us
I20260709 13:04:26.058527 15846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:26.058863 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:26.197400 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 138593us
I20260709 13:04:26.061031 19592 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:26.061363 19587 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:26.062732 19412 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:26.033644 19662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.007s	sys 0.026s
W20260709 13:04:26.118800 15054 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.020s	sys 0.014s
W20260709 13:04:26.295120 15054 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.020s	sys 0.015s
W20260709 13:04:26.295564 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 718485us
I20260709 13:04:26.297678 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.120225 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.023494 19671 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:26.029958 19664 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 13:04:26.057772 19599 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.671s	user 0.014s	sys 0.000s
I20260709 13:04:26.050019 19559 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:26.055469 19602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.015s	sys 0.018s
W20260709 13:04:26.151041 19728 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:04:26.151998 19740 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:26.051486 19471 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.171415 19614 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:26.051880 19237 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.049573 19655 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.338837 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:26.243940 19659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.005s	sys 0.023s
I20260709 13:04:26.246729 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.201167 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:26.189956 19477 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:26.285781 15847 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:26.342032 15200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.023s	sys 0.018s
W20260709 13:04:26.342779 15200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.023s	sys 0.018s
I20260709 13:04:26.343032 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.210011 19514 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.302124 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:26.206362 19595 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.246147 19564 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:26.298547 15085 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:26.206053 19573 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:26.309700 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.007s	sys 0.018s
W20260709 13:04:26.357718 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.007s	sys 0.018s
I20260709 13:04:26.318012 15080 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:04:26.299461 19662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.007s	sys 0.029s
I20260709 13:04:26.323429 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.292553 19588 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:26.304354 19420 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:04:26.367362 15232 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.009s	sys 0.013s
I20260709 13:04:26.328114 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.329241 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.333477 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:26.372028 15161 raft_consensus.cc:2410] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:04:26.334285 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 921) took 2012 ms. Trace:
W20260709 13:04:26.371637 15232 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.009s	sys 0.013s
W20260709 13:04:26.335196 15845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.005s	sys 0.012s
I20260709 13:04:26.333591 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:26.374646 15845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.005s	sys 0.013s
W20260709 13:04:26.298372 19702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.016s
W20260709 13:04:26.377804 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.023s
I20260709 13:04:26.338552 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:26.379555 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.000s	sys 0.024s
I20260709 13:04:26.379666 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:26.379916 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 505408us
I20260709 13:04:26.339126 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.339975 15156 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:26.259838 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.023s
I20260709 13:04:26.384860 15088 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 8: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:26.385004 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.006s	sys 0.023s
I20260709 13:04:26.338898 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.385648 15079 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
W20260709 13:04:26.341238 15842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.015s	sys 0.013s
I20260709 13:04:26.344117 15847 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:26.298867 15843 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:26.346769 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:26.324668 19371 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:26.353300 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 152257us
I20260709 13:04:26.363237 19747 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:26.359498 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57563.8us
W20260709 13:04:26.341873 19659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.006s	sys 0.023s
I20260709 13:04:26.338549 19454 tablet_copy_client.cc:538] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:26.375641 15163 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:04:26.374963 15084 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:26.340374 19518 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:04:26.497498 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.013s	sys 0.017s
W20260709 13:04:26.497949 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.013s	sys 0.017s
I20260709 13:04:26.452513 19703 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.372501 15237 rpcz_store.cc:276] 0709 13:04:24.322081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.322221 (+   140us) service_pool.cc:224] Handling call
0709 13:04:24.368007 (+ 45786us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:24.557121 (+189114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.557152 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:24.557799 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:04:25.187009 (+629210us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:25.187071 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.384680 (+197609us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b5400294d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:25.384723 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.307780 (+923057us) raft_consensus.cc:1685] finished
0709 13:04:26.307802 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.334205 (+ 26403us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":983,"raft.queue_time_us":34436,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":45181,"spinlock_wait_cycles":125489152,"thread_start_us":591805,"threads_started":1,"wal-append.queue_time_us":1117915}
I20260709 13:04:26.450937 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:26.387704 15083 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:26.502015 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2723) took 1480 ms. Trace:
I20260709 13:04:26.502583 15159 rpcz_store.cc:276] 0709 13:04:25.021580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.021703 (+   123us) service_pool.cc:224] Handling call
0709 13:04:25.022048 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.022441 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.022467 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.022958 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.672886 (+649928us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:25.672951 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.673050 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.501729 (+828679us) raft_consensus.cc:1685] finished
0709 13:04:26.501758 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.501945 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2683,"raft.queue_time_us":11500,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":19526,"thread_start_us":649430,"threads_started":1,"wal-append.queue_time_us":980070}
I20260709 13:04:26.378522 19573 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.460326 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:26.514267 14981 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.016s	sys 0.017s
I20260709 13:04:26.449651 15156 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:26.452149 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:26.498664 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 893364us
W20260709 13:04:26.413602 15842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.015s	sys 0.013s
W20260709 13:04:26.476979 15086 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.025s
W20260709 13:04:26.515906 15086 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.003s	sys 0.025s
I20260709 13:04:26.515998 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:26.340963 19602 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:26.340473 19553 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 13:04:26.506708 19730 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.468s	user 0.000s	sys 0.001s
I20260709 13:04:26.319195 19599 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:26.377691 19702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.011s	sys 0.016s
I20260709 13:04:26.365355 19614 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:26.377230 19476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.011s	sys 0.010s
I20260709 13:04:26.347163 19587 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
W20260709 13:04:26.319295 19548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.022s	sys 0.030s
W20260709 13:04:26.565346 19548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.022s	sys 0.030s
I20260709 13:04:26.484565 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:26.487470 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.380443 19564 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:26.482992 19524 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:26.478535 19452 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:04:26.516541 15846 raft_consensus.cc:606] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:26.500859 19592 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:26.520390 15156 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:26.452322 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.460807 15157 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:04:26.522451 14981 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.017s	sys 0.017s
I20260709 13:04:26.525279 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.526552 15842 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:26.546456 19476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.012s	sys 0.010s
I20260709 13:04:26.458875 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:26.504436 19471 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:26.491024 19371 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.586231 15846 raft_consensus.cc:614] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:26.540335 19614 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.505151 19703 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:26.606683 19739 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:04:26.607785 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 942) took 1120 ms. Trace:
W20260709 13:04:26.608300 19746 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:04:26.609411 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1993) took 1606 ms. Trace:
I20260709 13:04:26.609694 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 922) took 1466 ms. Trace:
I20260709 13:04:26.610039 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1990) took 1610 ms. Trace:
I20260709 13:04:26.611445 19514 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:26.610153 15158 rpcz_store.cc:276] 0709 13:04:25.143647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.387411 (+243764us) service_pool.cc:224] Handling call
0709 13:04:25.387727 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.425961 (+ 38234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.425996 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.429209 (+  3213us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.997213 (+568004us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:25.997276 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.997382 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.609497 (+612115us) raft_consensus.cc:1685] finished
0709 13:04:26.609518 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.609638 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1624,"raft.queue_time_us":37838,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":16758,"thread_start_us":567618,"threads_started":1,"wal-append.queue_time_us":949302}
W20260709 13:04:26.611724 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122917us
W20260709 13:04:26.613303 19752 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:04:26.613894 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:26.614629 19599 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:26.615610 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.616077 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.617170 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1126) took 1021 ms. Trace:
I20260709 13:04:26.610545 15228 rpcz_store.cc:276] 0709 13:04:24.999507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.031281 (+ 31774us) service_pool.cc:224] Handling call
0709 13:04:25.031576 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.037291 (+  5715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.037323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.059763 (+ 22440us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.903389 (+843626us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:25.903434 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.903502 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.609849 (+706347us) raft_consensus.cc:1685] finished
0709 13:04:26.609869 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.609989 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25945,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7100,"spinlock_wait_cycles":139136,"thread_start_us":843172,"threads_started":1,"wal-append.queue_time_us":1267680}
W20260709 13:04:26.591430 19416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.020s	sys 0.029s
I20260709 13:04:26.585788 19671 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:26.609786 15236 rpcz_store.cc:276] 0709 13:04:25.002710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.002852 (+   142us) service_pool.cc:224] Handling call
0709 13:04:25.003127 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.003341 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:25.669375 (+666034us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.609046 (+939671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.609108 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:26.609154 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.609227 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.609347 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223550,"prepare.queue_time_us":14859,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":29,"raft.queue_time_us":224296,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"thread_start_us":1380936,"threads_started":2,"wal-append.queue_time_us":883713}
I20260709 13:04:26.581055 19412 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:26.622712 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 138244us
W20260709 13:04:26.604012 19733 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:04:26.601722 19614 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:26.608235 15232 rpcz_store.cc:276] 0709 13:04:25.487600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.487969 (+   369us) service_pool.cc:224] Handling call
0709 13:04:25.488291 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.488520 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:25.488780 (+   260us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.378803 (+890023us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.607217 (+228414us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b18003e3680. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:26.607401 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.607454 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.607509 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.607588 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.607720 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140100,"prepare.queue_time_us":8404,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":107,"raft.queue_time_us":351712,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":37828,"spinlock_wait_cycles":4028672,"thread_start_us":848293,"threads_started":1}
I20260709 13:04:26.634482 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:26.617586 15840 rpcz_store.cc:276] 0709 13:04:25.596184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.597955 (+  1771us) service_pool.cc:224] Handling call
0709 13:04:25.598415 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.598894 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.598942 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.610583 (+ 11641us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.048371 (+437788us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:26.048421 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.048506 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.616956 (+568450us) raft_consensus.cc:1685] finished
0709 13:04:26.616981 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.617113 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157,"raft.queue_time_us":11116,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":30947,"thread_start_us":356543,"threads_started":1,"wal-append.queue_time_us":778724}
I20260709 13:04:26.582520 19600 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:26.597003 19497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.015s	sys 0.026s
I20260709 13:04:26.603725 19704 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:26.600827 19592 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:26.678524 19592 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.523476 19588 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:26.629359 19416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.020s	sys 0.029s
W20260709 13:04:26.670073 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 353816us
W20260709 13:04:26.634078 19745 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
W20260709 13:04:26.669056 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 348633us
I20260709 13:04:26.672380 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1161) took 1620 ms. Trace:
I20260709 13:04:26.690424 15839 rpcz_store.cc:276] 0709 13:04:25.051807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.051939 (+   132us) service_pool.cc:224] Handling call
0709 13:04:25.052321 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.052553 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.052575 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.063895 (+ 11320us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.677401 (+613506us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:25.677460 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.677556 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.672080 (+994524us) raft_consensus.cc:1685] finished
0709 13:04:26.672111 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.672295 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":592585,"threads_started":1,"wal-append.queue_time_us":917085}
I20260709 13:04:26.617569 19576 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:26.692543 19576 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:26.677448 19497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.015s	sys 0.027s
I20260709 13:04:26.694283 19497 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:26.695451 19497 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:26.700557 19420 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:26.702754 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:26.616787 19599 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:04:26.701998 19741 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.188s	user 0.003s	sys 0.001s
I20260709 13:04:26.728320 19526 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:26.730818 19526 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.674371 19655 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:26.761750 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 415009us
I20260709 13:04:26.673398 19760 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:26.763371 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:26.704531 19595 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:26.665874 19412 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.702109 19572 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.703989 19708 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:26.721097 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:26.706162 19757 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:26.712415 19416 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:26.757316 19734 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:04:26.795207 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1093) took 1644 ms. Trace:
I20260709 13:04:26.719223 19518 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:26.769889 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 282413us
I20260709 13:04:26.754350 19663 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:26.855903 15086 rpcz_store.cc:276] 0709 13:04:25.151343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.203021 (+ 51678us) service_pool.cc:224] Handling call
0709 13:04:25.203283 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.203487 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:25.203815 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.516284 (+1312469us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.794531 (+278247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.794612 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:26.794678 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.794775 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.794950 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":864226,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":10742,"raft.queue_time_us":211538,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":1589752,"threads_started":3,"wal-append.queue_time_us":1157369}
I20260709 13:04:26.874354 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 920) took 1822 ms. Trace:
I20260709 13:04:26.874423 15160 raft_consensus.cc:1217] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.7->[7.8-7.8]   Dedup: 7.8->[]
I20260709 13:04:26.875128 15160 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:04:26.717896 19471 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
W20260709 13:04:26.879186 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 363248us
I20260709 13:04:26.770164 19760 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.874991 15154 rpcz_store.cc:276] 0709 13:04:25.052464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.052949 (+   485us) service_pool.cc:224] Handling call
0709 13:04:25.053244 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:25.060476 (+  7232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.060503 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:25.067414 (+  6911us) log.cc:844] Serialized 52 byte log entry
0709 13:04:25.798343 (+730929us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:25.798384 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.798452 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.874091 (+1075639us) raft_consensus.cc:1685] finished
0709 13:04:26.874119 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.874295 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5946,"raft.queue_time_us":9487,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":1876,"thread_start_us":676367,"threads_started":1,"wal-append.queue_time_us":952717}
I20260709 13:04:26.851085 19671 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:26.885666 19697 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.927s	user 0.011s	sys 0.002s
I20260709 13:04:26.706635 19564 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.897321 19452 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.852710 19572 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.901197 19760 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:26.940395 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.010s	sys 0.015s
I20260709 13:04:26.781245 19595 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:26.948870 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.855093 19708 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:26.966784 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.010s	sys 0.015s
I20260709 13:04:26.886665 19769 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.991766 15088 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:26.934451 19697 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:26.993302 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.002934 15088 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:26.983521 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.880256 19518 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:26.998369 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:26.999642 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.001040 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.006372 15079 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:27.021824 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.022495 15086 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:27.023353 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:26.969161 19598 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.183s	user 0.013s	sys 0.001s
I20260709 13:04:26.932819 19573 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.027714 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.028476 15086 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:27.029007 15083 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:26.940783 19702 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:26.984599 19754 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 13:04:26.941967 19773 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:27.030791 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.924165 19768 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:04:27.035635 15080 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:27.034768 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:27.047135 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:27.032933 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:26.928323 19671 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:26.997376 19602 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.061666 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.062397 15079 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:27.060832 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.073251 15083 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:26.971150 19452 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:27.054581 19741 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:27.091019 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43673.1us
I20260709 13:04:27.028394 19703 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:27.146090 19763 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.375s	user 0.000s	sys 0.000s
W20260709 13:04:27.146930 19771 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:04:27.010285 19769 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.147351 19454 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap starting.
I20260709 13:04:27.048028 19773 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.154058 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:27.049098 19416 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.088692 19602 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.148000 19784 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.155753 15083 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:04:27.194424 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58336.2us
W20260709 13:04:27.220080 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66131.4us
I20260709 13:04:27.243547 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:27.243549 15163 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.012s	sys 0.038s
W20260709 13:04:27.245911 15163 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.013s	sys 0.038s
I20260709 13:04:27.285182 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:27.286285 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48874.9us
I20260709 13:04:27.287657 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:27.195031 19765 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.445s	user 0.001s	sys 0.000s
W20260709 13:04:27.229522 19758 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.524s	user 0.000s	sys 0.001s
W20260709 13:04:27.203933 19761 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:04:27.235126 19476 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.043s	sys 0.056s
I20260709 13:04:27.312336 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 930) took 1494 ms. Trace:
I20260709 13:04:27.209889 19572 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.044373 19598 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:27.325194 15814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.032s
I20260709 13:04:27.204173 19755 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.213177 19762 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:27.328150 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.995s	user 0.031s	sys 0.064s
I20260709 13:04:27.330580 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:27.356169 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.032s	sys 0.064s
W20260709 13:04:27.356580 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.0302e+06us
I20260709 13:04:27.357525 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.221737 19526 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:27.229557 19614 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.005s
I20260709 13:04:27.219566 19708 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.220429 19770 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.210762 19518 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.232206 19773 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:27.049494 19702 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.230638 19412 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.157125 19780 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.377523 19780 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:27.178568 19477 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.003s
I20260709 13:04:27.379134 19477 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.380342 15087 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:27.381448 19477 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.341418 15163 rpcz_store.cc:276] 0709 13:04:25.818381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.495345 (+676964us) service_pool.cc:224] Handling call
0709 13:04:26.495664 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.495889 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:26.722549 (+226660us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.286720 (+564171us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.311922 (+ 25202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.311986 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:27.312031 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.312103 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.312258 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39589,"prepare.queue_time_us":506413,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":6554,"raft.queue_time_us":90223,"thread_start_us":740191,"threads_started":2}
I20260709 13:04:27.228951 19371 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.197923 19775 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:27.383590 19775 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.384027 19775 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:27.349263 15814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.009s	sys 0.032s
I20260709 13:04:27.194702 19600 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.147s
W20260709 13:04:27.341219 19476 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.044s	sys 0.056s
I20260709 13:04:27.242367 19416 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.388036 19416 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.365545 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.369860 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:27.355767 19356 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.374s	user 0.012s	sys 0.001s
I20260709 13:04:27.389676 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.389639 19356 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.392104 19356 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.372408 15844 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:04:27.385068 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 664053us
I20260709 13:04:27.393041 15162 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:27.393152 19371 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.393890 19371 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:27.369849 19614 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.396507 15235 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:27.397534 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.342274 19697 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:04:27.398324 15083 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:27.228770 19784 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.362059 19755 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.402206 19697 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:27.402536 19784 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.403340 19755 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.404330 19671 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.405639 19775 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.355177 19762 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:27.392618 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:27.372215 19526 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.411136 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.428171 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.416193 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:27.429020 15085 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:27.422989 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.425539 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:27.413126 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 945) took 1924 ms. Trace:
I20260709 13:04:27.430873 15848 raft_consensus.cc:2376] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11. Re-sending same reply.
I20260709 13:04:27.426267 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:27.430907 15234 rpcz_store.cc:276] 0709 13:04:25.488534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.806369 (+317835us) service_pool.cc:224] Handling call
0709 13:04:25.838558 (+ 32189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.838742 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.838777 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.839197 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.348802 (+509605us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:26.348878 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.348971 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.412917 (+1063946us) raft_consensus.cc:1685] finished
0709 13:04:27.412937 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.413070 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":445873,"threads_started":1,"wal-append.queue_time_us":806520}
I20260709 13:04:27.423427 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:27.410128 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.426982 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:27.433200 15840 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:04:27.424224 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:27.433410 15843 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.417265 15088 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:04:27.444882 15813 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.502s (SENT)
I20260709 13:04:27.445559 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:27.452461 19703 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:27.455510 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:27.466311 19794 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:27.467316 19783 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:27.468041 19753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.031s
W20260709 13:04:27.468492 19753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.010s	sys 0.031s
I20260709 13:04:27.417590 19777 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:27.402594 19697 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:27.419500 19602 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.481761 15843 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:27.408831 19775 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.453356 19614 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:27.479334 19738 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.290s	user 0.010s	sys 0.013s
I20260709 13:04:27.479008 19595 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:27.476492 19783 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:27.471590 19477 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:04:27.489816 19777 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:27.503168 19452 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.517848 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.520779 15081 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:27.523301 19769 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.484149 19760 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.409510 19762 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.538931 19702 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.541591 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 1994) took 2537 ms. Trace:
W20260709 13:04:27.541860 19759 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.931s	user 0.008s	sys 0.019s
W20260709 13:04:27.546430 15841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.010s	sys 0.017s
W20260709 13:04:27.546790 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 385790us
I20260709 13:04:27.526021 19477 consensus_queue.cc:1243] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:04:27.550567 15841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.010s	sys 0.018s
I20260709 13:04:27.552301 19773 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.555423 19773 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.568032 15085 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:27.547091 15230 rpcz_store.cc:276] 0709 13:04:25.004362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.004521 (+   159us) service_pool.cc:224] Handling call
0709 13:04:25.004886 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.033032 (+ 28146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.033076 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.033568 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.797172 (+763604us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:25.797488 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 13:04:25.797787 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.797860 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.340965 (+1543105us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:27.541207 (+200242us) raft_consensus.cc:1685] finished
0709 13:04:27.541355 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.541521 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8373,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":131,"raft.queue_time_us":63272,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":40107,"spinlock_wait_cycles":88337792,"thread_start_us":763300,"threads_started":1,"wal-append.queue_time_us":1015121}
W20260709 13:04:27.585796 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.588397 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:27.589989 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:27.593478 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 140434us
W20260709 13:04:27.594453 15813 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.651s (SENT)
I20260709 13:04:27.569589 19782 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.594957 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:27.580717 19762 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.611485 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:27.527266 19697 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.615350 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1180) took 2118 ms. Trace:
I20260709 13:04:27.615935 15845 rpcz_store.cc:276] 0709 13:04:25.496750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.536991 (+ 40241us) service_pool.cc:224] Handling call
0709 13:04:25.537479 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.538484 (+  1005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.538533 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.545540 (+  7007us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.375241 (+829701us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:26.375284 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.375352 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.541075 (+1165723us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:27.614957 (+ 73882us) raft_consensus.cc:1685] finished
0709 13:04:27.614989 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.615275 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10005376,"thread_start_us":742103,"threads_started":1,"wal-append.queue_time_us":1024526}
I20260709 13:04:27.614681 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.622076 15158 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:27.622802 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.544866 19783 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.627931 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.631505 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 935) took 1296 ms. Trace:
I20260709 13:04:27.617213 19773 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.635794 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:27.638792 15163 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:27.621013 19476 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:04:27.630086 19809 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.600801 19782 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.636117 19471 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:27.634434 19735 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:27.633194 15159 rpcz_store.cc:276] 0709 13:04:26.331078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.506304 (+175226us) service_pool.cc:224] Handling call
0709 13:04:26.506657 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.506898 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:26.940898 (+434000us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.367264 (+426366us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.626692 (+259428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.627099 (+   407us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:27.627179 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.627279 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.627456 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516853,"prepare.queue_time_us":557897,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":30561,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":258888,"spinlock_wait_cycles":23217280,"thread_start_us":1077850,"threads_started":3,"wal-append.queue_time_us":573365}
I20260709 13:04:27.595391 19777 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.648509 19518 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:27.635898 19743 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:27.631733 19788 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:27.634138 19452 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:27.629972 19799 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:04:27.652251 19735 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:27.650403 19471 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:27.640704 19812 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:04:27.657593 19800 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:04:27.677181 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:27.656598 19815 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.684670 19412 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:27.686480 19802 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:04:27.687317 19808 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:04:27.688216 19805 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:04:27.678769 19798 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:04:27.651402 19809 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.678047 19735 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:27.700935 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.06651e+06us
I20260709 13:04:27.701818 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.702605 19809 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.704015 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:35386 (request call id 5) took 1645 ms. Trace:
I20260709 13:04:27.704484 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1184) took 1448 ms. Trace:
I20260709 13:04:27.704752 19799 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:27.705204 19704 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:27.706055 19704 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.705098 15841 rpcz_store.cc:276] 0709 13:04:26.256407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.567978 (+311571us) service_pool.cc:224] Handling call
0709 13:04:26.568244 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.568597 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.568629 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.569116 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.551041 (+981925us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:27.551091 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.551163 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.704274 (+153111us) raft_consensus.cc:1685] finished
0709 13:04:27.704296 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.704426 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5201,"raft.queue_time_us":7103,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7207,"thread_start_us":981566,"threads_started":1,"wal-append.queue_time_us":1043229}
I20260709 13:04:27.678615 19471 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.707199 19788 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.707710 19788 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.710069 19812 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.710645 19812 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:27.697885 19804 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:04:27.699529 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 944) took 2211 ms. Trace:
I20260709 13:04:27.699666 19813 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.692992 19815 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:27.712138 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 948846us
I20260709 13:04:27.680421 19760 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:27.679725 19743 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:27.720511 19655 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:27.721714 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.704412 15846 rpcz_store.cc:276] 0709 13:04:26.058228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.058395 (+   167us) service_pool.cc:224] Handling call
0709 13:04:27.703829 (+1645434us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:27.703951 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734225280}
W20260709 13:04:27.723541 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 688767us
I20260709 13:04:27.738698 15083 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:27.713686 15233 rpcz_store.cc:276] 0709 13:04:25.488245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.644455 (+156210us) service_pool.cc:224] Handling call
0709 13:04:25.680465 (+ 36010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.189753 (+509288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.189805 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.190485 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.635503 (+445018us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.638846 (+  3343us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.698883 (+1060037us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b18003e35c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:27.699078 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.699157 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.699206 (+    49us) raft_consensus.cc:1685] finished
0709 13:04:27.699312 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.699464 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68092,"prepare.queue_time_us":2127,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2776,"raft.queue_time_us":1204075,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":5094,"spinlock_wait_cycles":10527104,"thread_start_us":943930,"threads_started":2,"wal-append.queue_time_us":705850}
I20260709 13:04:27.692605 19602 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:27.740190 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.740677 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.741094 15086 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:27.741607 15084 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:27.736984 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1105) took 1426 ms. Trace:
I20260709 13:04:27.690598 19412 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.742092 15087 rpcz_store.cc:276] 0709 13:04:26.311361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.323117 (+ 11756us) service_pool.cc:224] Handling call
0709 13:04:27.736715 (+1413598us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001ff4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:27.736913 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343910144}
I20260709 13:04:27.743185 15080 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [7.8-12.10]
I20260709 13:04:27.739074 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:27.741555 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.745656 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:27.746459 15086 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:27.747377 15086 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:27.747686 15086 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:27.743659 15087 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: [7.8-10.9]
W20260709 13:04:27.750856 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49145.3us
I20260709 13:04:27.733767 19559 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:27.751791 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:27.749845 19760 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.746549 19412 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.750036 19602 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.760730 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 954) took 1407 ms. Trace:
W20260709 13:04:27.760941 19803 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 13:04:27.792306 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43913.8us
W20260709 13:04:27.795262 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.761649 19659 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:27.800460 19659 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:27.801455 19659 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.705367 19799 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.802644 19659 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:27.804026 19471 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.749671 15086 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.799609 15237 rpcz_store.cc:276] 0709 13:04:26.353550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.512265 (+158715us) service_pool.cc:224] Handling call
0709 13:04:26.512580 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.558244 (+ 45664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.558290 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.558355 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:26.558401 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.760402 (+1202001us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b5400177854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:27.760462 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.760654 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73211136}
I20260709 13:04:27.765029 19704 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:27.766218 19780 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:27.772359 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:27.846073 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73813.7us
I20260709 13:04:27.847031 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:27.740649 19655 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:27.806439 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54743.6us
I20260709 13:04:27.793762 19760 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.740141 19743 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.732609 19813 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.793591 19818 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.805222 19753 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:27.776319 19599 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.834143 15086 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:27.763516 19744 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.158s	user 0.005s	sys 0.018s
W20260709 13:04:27.835640 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83799.6us
I20260709 13:04:27.865016 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.838356 15156 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.851446 19655 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.841213 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:27.835681 19704 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.804471 19799 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:27.842289 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:27.769604 19559 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.775851 19584 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.817463 19708 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.835749 19770 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.866494 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:27.840858 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:27.841560 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:27.886803 15156 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:27.849428 19664 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:27.849524 19476 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:27.874791 15155 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.876886 15085 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:27.882290 15157 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.884951 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:27.888967 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:27.876256 19704 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:27.861483 19773 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:27.883406 19559 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:27.865214 19743 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:27.883553 19747 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:27.862598 19420 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:04:27.913683 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:27.925952 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.870591 19599 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.888514 19584 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.933140 15233 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:27.934461 19584 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:27.934949 19801 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 13:04:27.935586 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.936617 19735 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.938050 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.939819 15236 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:27.909298 19780 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:27.939713 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.904460 19564 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:27.866262 19818 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.942369 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.952373 15228 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:27.953685 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.888068 19572 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:04:27.945407 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.956367 15230 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:27.957577 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.958693 15230 raft_consensus.cc:2410] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:27.960134 19747 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.960613 15233 pending_rounds.cc:85] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 7
I20260709 13:04:27.961036 15233 pending_rounds.cc:107] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:27.961418 15233 raft_consensus.cc:2889] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:27.960880 19747 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:27.921933 19664 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:27.891619 19770 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:27.984520 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.986385 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.929876 19708 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:27.974069 15155 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:27.976181 15086 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:04:27.998631 15086 pending_rounds.cc:85] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 10
I20260709 13:04:27.999094 15086 pending_rounds.cc:107] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:04:27.999431 15086 raft_consensus.cc:2889] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.001369 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:27.976773 15237 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:27.983058 15229 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:27.971956 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.007406 15228 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:28.008888 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.009842 15228 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:27.943637 19584 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.004921 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:27.995779 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.943905 19780 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:28.019559 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:28.011189 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:28.023815 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.026189 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.026892 15234 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:28.020056 15230 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:28.030055 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.974838 19664 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:28.033721 15159 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:27.984346 19702 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:27.974792 19747 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.035307 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:27.983764 19564 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:27.983093 19812 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.034657 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:28.085345 15229 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:28.027498 19822 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 13:04:28.041595 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:27.997129 19815 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.025988 19780 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.027062 19783 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:28.087352 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.096460 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.097236 15237 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:28.084015 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.090772 15230 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:28.093545 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:28.012271 19799 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:04:28.116410 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:28.121074 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.121942 15230 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:28.123031 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.118436 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.123859 15237 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:28.123989 15086 pending_rounds.cc:85] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 8
I20260709 13:04:28.124348 15086 pending_rounds.cc:107] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:28.124579 15086 raft_consensus.cc:2889] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.022228 19596 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:04:28.117357 15234 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:28.126425 15230 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:04:28.130151 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.134557 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.026254 19584 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.127403 19526 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:28.138993 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:28.140326 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:28.121469 19762 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:04:28.139218 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:28.123160 19783 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:28.143327 19772 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:28.143151 19815 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:28.143549 19743 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.143870 19718 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:28.087103 19702 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.144565 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:28.136729 19602 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.145637 15843 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:28.146545 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.146638 19655 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.147744 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.148700 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.037886 19823 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:04:28.126523 19799 consensus_queue.cc:1243] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:28.122309 19452 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:28.113795 19471 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:28.160683 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:28.223989 15234 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:28.162353 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.162477 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.163619 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:28.171530 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:28.229274 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57811us
I20260709 13:04:28.231045 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:28.145228 19526 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:28.176653 15237 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:04:28.233125 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.115734 19788 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.188560 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.190603 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.150121 19735 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:28.245802 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.247251 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.226513 15848 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:04:28.239001 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.265493 15845 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:28.265547 15237 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:28.248315 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.170033 19718 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:28.097669 19564 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.263835 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.269774 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:28.270639 15840 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:28.219158 19757 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.219412 19812 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:28.307271 15230 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:28.310079 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:28.310979 15230 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:28.235548 19815 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:28.316134 19757 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:28.176651 19743 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.150458 19664 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.326824 15846 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:28.236889 19526 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.325325 19743 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:28.325395 19827 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:04:28.340821 19824 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.321369 19770 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.341553 19824 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.316196 19599 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.324920 19812 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.343158 19812 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:28.344343 19824 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.344609 19237 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.347183 19704 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:28.317341 19829 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:04:28.318759 19757 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:28.168083 19772 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.327600 15234 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:28.352411 15083 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:28.356441 15237 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:28.365418 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2726) took 3251 ms. Trace:
I20260709 13:04:28.377185 15160 rpcz_store.cc:276] 0709 13:04:25.109255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.109411 (+   156us) service_pool.cc:224] Handling call
0709 13:04:26.874282 (+1764871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.878359 (+  4077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.360578 (+1482219us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54001c6bd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:28.360753 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4057984768}
I20260709 13:04:28.309337 19813 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.316809 19815 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:28.365641 19412 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.001s
W20260709 13:04:28.370788 19830 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:28.367256 19697 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.367652 19576 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.353103 19526 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.369905 19655 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:28.405216 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.385109 19818 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.397316 19664 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:28.395742 19412 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:28.421303 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 286957us
I20260709 13:04:28.424551 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:28.425395 15157 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.328459 19783 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.441744 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:28.385941 19718 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:28.428404 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 451120us
I20260709 13:04:28.462960 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:28.424829 19788 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:28.454526 19588 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.003s	sys 0.027s
I20260709 13:04:28.453495 19420 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:28.466347 19356 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.468503 19584 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.476111 19718 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:28.509919 19832 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:04:28.511600 19831 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:04:28.513746 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:28.515093 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:28.515764 19659 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.713s	user 0.001s	sys 0.008s
I20260709 13:04:28.451929 19773 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:04:28.524075 15842 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:28.484035 19704 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.485947 19782 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.515713 15156 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:28.530802 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:28.479136 19588 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.004s	sys 0.027s
I20260709 13:04:28.482975 19420 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.528266 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 950) took 2175 ms. Trace:
I20260709 13:04:28.532618 15235 rpcz_store.cc:276] 0709 13:04:26.352561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.485193 (+132632us) service_pool.cc:224] Handling call
0709 13:04:26.500127 (+ 14934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.412515 (+912388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.519617 (+1107102us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54000acdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:28.528169 (+  8552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060979584}
I20260709 13:04:28.494508 19702 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.494210 15157 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:28.539755 19518 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:28.540295 19518 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.541294 15235 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:28.549014 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.549834 15156 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:28.550992 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.552557 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.553980 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.557803 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:28.533270 19420 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:28.546147 19748 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 2.023s	user 0.009s	sys 0.009s
I20260709 13:04:28.538667 19772 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.518579 19356 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.577093 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:28.567935 19775 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.567915 19497 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.583389 19772 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:28.599944 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:28.619704 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 184792us
W20260709 13:04:28.605957 19834 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:04:28.602742 19772 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.609148 19773 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:28.610589 19412 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.594033 19775 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.630440 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:28.594550 19497 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.640100 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:28.643749 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:28.644526 15844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1128) took 3025 ms (client timeout 2764 ms). Trace:
W20260709 13:04:28.662051 15844 rpcz_store.cc:269] 0709 13:04:25.614610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.614790 (+   180us) service_pool.cc:224] Handling call
0709 13:04:28.639858 (+3025068us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54007c4554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:28.640017 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4038709120}
I20260709 13:04:28.626080 19772 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:28.646327 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.632192 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:28.649324 15843 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:28.646548 19842 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.645149 19756 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:28.643512 19773 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:28.648427 19839 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:28.670542 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 936) took 3442 ms. Trace:
I20260709 13:04:28.674747 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.675323 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.676748 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2027) took 1307 ms. Trace:
I20260709 13:04:28.625051 19846 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:04:28.677142 15082 rpcz_store.cc:276] 0709 13:04:27.369336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.370478 (+  1142us) service_pool.cc:224] Handling call
0709 13:04:27.373812 (+  3334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.374197 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.374232 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.374697 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.920694 (+545997us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b180027a8a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9
0709 13:04:28.223760 (+303066us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:28.223817 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.223890 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.670494 (+446604us) raft_consensus.cc:1685] finished
0709 13:04:28.670513 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.676684 (+  6171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2416,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":40069504,"thread_start_us":302858,"threads_started":1,"wal-append.queue_time_us":1102669}
I20260709 13:04:28.678210 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:28.676095 15160 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:28.682343 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.683533 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:28.676327 19839 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.684232 15160 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:28.684300 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:28.668176 19412 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:28.672601 15231 rpcz_store.cc:276] 0709 13:04:25.228099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.228246 (+   147us) service_pool.cc:224] Handling call
0709 13:04:25.261602 (+ 33356us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:25.278895 (+ 17293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.278939 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:25.339231 (+ 60292us) log.cc:844] Serialized 78 byte log entry
0709 13:04:25.986973 (+647742us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.023863 (+ 36890us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.038054 (+ 14191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.038142 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.525600 (+2487458us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:28.666805 (+141205us) raft_consensus.cc:1685] finished
0709 13:04:28.670298 (+  3493us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.670482 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30610,"prepare.queue_time_us":36586,"prepare.run_cpu_time_us":765,"prepare.run_wall_time_us":6374,"raft.queue_time_us":124863,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":10277,"spinlock_wait_cycles":44702464,"thread_start_us":453245,"threads_started":1,"wal-append.queue_time_us":999777}
I20260709 13:04:28.686276 15814 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.685230 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:28.681274 19835 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:04:28.654736 19812 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.672816 19756 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:28.689750 15160 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:04:28.696242 15160 pending_rounds.cc:85] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 8
I20260709 13:04:28.696604 15160 pending_rounds.cc:107] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:28.696869 15160 raft_consensus.cc:2889] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:28.685582 19837 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:04:28.701010 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:28.704061 19825 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.799s	user 0.000s	sys 0.003s
I20260709 13:04:28.705513 19760 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.707509 19773 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.705899 19497 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.686877 19659 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:28.716212 15160 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:28.689949 19513 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:28.691401 19237 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:28.720268 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.686024 19839 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:28.721557 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.696200 19756 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:28.724288 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.725549 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.708716 19518 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.728305 15052 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:28.729177 19799 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:28.738811 19799 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:28.739353 19799 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.731065 19813 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:28.714643 19783 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:28.731274 19587 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.722189 19853 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.002s
I20260709 13:04:28.732373 19775 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:28.733422 19849 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:28.734292 15843 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:04:28.727808 19793 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.188s	user 0.001s	sys 0.005s
W20260709 13:04:28.738173 19848 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:04:28.682914 19842 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:28.722711 19851 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
I20260709 13:04:28.742280 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:28.748800 19371 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:28.749754 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.720181 19513 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:28.726758 19818 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:28.757365 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:28.758782 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2227) took 3159 ms. Trace:
I20260709 13:04:28.783571 15162 rpcz_store.cc:276] 0709 13:04:25.599479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.599640 (+   161us) service_pool.cc:224] Handling call
0709 13:04:25.600032 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.393487 (+1793455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.758475 (+1364988us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400086bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:28.758703 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4123145088}
I20260709 13:04:28.736864 19356 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.764647 19842 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.765887 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1124) took 1022 ms. Trace:
I20260709 13:04:28.789795 15086 rpcz_store.cc:276] 0709 13:04:27.743791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.088311 (+344520us) service_pool.cc:224] Handling call
0709 13:04:28.123819 (+ 35508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.125150 (+  1331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.125179 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.125590 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.481388 (+355798us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:28.481423 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.481488 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.765647 (+284159us) raft_consensus.cc:1685] finished
0709 13:04:28.765670 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.765828 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":337248,"threads_started":1,"wal-append.queue_time_us":451268}
I20260709 13:04:28.752898 19587 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:28.771086 19559 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.844s	user 0.006s	sys 0.006s
I20260709 13:04:28.801077 19559 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:28.802490 15234 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:28.805117 19818 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:28.752930 19783 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:28.806368 19559 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:28.805943 19818 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:04:28.811332 19780 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.005s
I20260709 13:04:28.811774 19780 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:28.817894 15082 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:28.819131 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:28.819933 15080 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:28.820962 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:28.821799 15083 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:04:28.783488 19513 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:28.825062 19860 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.148s	user 0.000s	sys 0.003s
I20260709 13:04:28.825318 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1123) took 1081 ms. Trace:
I20260709 13:04:28.825730 15087 rpcz_store.cc:276] 0709 13:04:27.743454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.750049 (+  6595us) service_pool.cc:224] Handling call
0709 13:04:27.750400 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.771331 (+ 20931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.771380 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.771435 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:27.771487 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.825093 (+1053606us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54003250d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:28.825140 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.825266 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46259968}
I20260709 13:04:28.826972 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:28.828399 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:28.775593 19670 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:28.776386 15155 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:28.753453 19813 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:28.797382 19476 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:28.799575 19526 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.829249 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1128) took 1058 ms. Trace:
I20260709 13:04:28.850080 15088 rpcz_store.cc:276] 0709 13:04:27.770675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.156570 (+385895us) service_pool.cc:224] Handling call
0709 13:04:28.156952 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.157298 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.157351 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.169653 (+ 12302us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.572775 (+403122us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:28.572888 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.573004 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.829022 (+256018us) raft_consensus.cc:1685] finished
0709 13:04:28.829043 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.829188 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1699,"raft.queue_time_us":10528,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15640,"thread_start_us":306777,"threads_started":1,"wal-append.queue_time_us":427871}
I20260709 13:04:28.829603 19703 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.857713 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1115) took 1116 ms. Trace:
I20260709 13:04:28.830612 19810 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:28.774171 19518 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.838297 15162 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:28.858316 15081 rpcz_store.cc:276] 0709 13:04:27.741010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.741432 (+   422us) service_pool.cc:224] Handling call
0709 13:04:27.741729 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.741933 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.741958 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.742367 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.834975 (+ 92608us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:27.865857 (+ 30882us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.866762 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.866829 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.703466 (+836637us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:28.835300 (+131834us) raft_consensus.cc:1685] finished
0709 13:04:28.857426 (+ 22126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.857633 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5185,"prepare.queue_time_us":63764,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":2975,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9195,"spinlock_wait_cycles":6501760,"thread_start_us":122828,"threads_started":2,"wal-append.queue_time_us":156347}
I20260709 13:04:28.867772 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:28.877704 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 144421us
I20260709 13:04:28.872596 15841 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:28.883317 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:28.886291 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:28.895665 15079 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:28.846993 19670 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:28.900548 15081 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:28.901873 15086 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:28.903355 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:28.905601 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1114) took 1165 ms. Trace:
I20260709 13:04:28.934840 15085 rpcz_store.cc:276] 0709 13:04:27.740637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.741328 (+   691us) service_pool.cc:224] Handling call
0709 13:04:28.905394 (+1164066us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540011add4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:28.905541 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302949760}
W20260709 13:04:28.906443 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.859229 19476 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:28.906852 19718 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:04:28.904801 19809 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.923601 19783 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.925251 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:28.926028 19587 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.926081 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2249) took 1085 ms. Trace:
I20260709 13:04:28.947896 15159 rpcz_store.cc:276] 0709 13:04:27.840447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.840575 (+   128us) service_pool.cc:224] Handling call
0709 13:04:28.925905 (+1085330us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:28.925998 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336050048}
W20260709 13:04:28.929189 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.863986 19810 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:28.867208 19659 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.936573 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:28.906651 19846 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.910315 19817 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.177s	user 0.002s	sys 0.000s
I20260709 13:04:28.879009 19703 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:28.950582 19786 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.410s	user 0.020s	sys 0.028s
I20260709 13:04:28.941154 19799 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.946928 19598 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.969137 19578 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:28.946141 19670 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:28.976603 19786 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.437s	user 0.021s	sys 0.028s
I20260709 13:04:28.953267 19686 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:04:28.988781 19670 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:28.984611 19578 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:28.956825 19659 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.998296 19513 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:28.999989 19513 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:28.959647 19497 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:04:28.983496 19476 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.003806 19772 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.006420 19420 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.007318 19772 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.961577 19862 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:04:28.950834 19809 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:28.984570 19697 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.003s
I20260709 13:04:29.013126 19809 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.959719 19846 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.044040 19420 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.000013 19659 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:29.008530 19839 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.042624 19780 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:28.961695 19813 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.057523 19846 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:28.938546 15081 raft_consensus.cc:606] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:28.964206 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 220544us
I20260709 13:04:29.115420 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.152918 15159 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.134502 15085 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: [7.8-10.9]
I20260709 13:04:29.098542 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 937) took 1511 ms. Trace:
I20260709 13:04:29.102099 15161 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:29.156338 15154 rpcz_store.cc:276] 0709 13:04:27.587189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.587332 (+   143us) service_pool.cc:224] Handling call
0709 13:04:27.587741 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.588272 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.588319 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.594908 (+  6589us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.642659 (+ 47751us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:27.642701 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.642778 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.909738 (+1266960us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:29.096695 (+186957us) raft_consensus.cc:1685] finished
0709 13:04:29.096781 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.098284 (+  1503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":967155,"raft.queue_time_us":112,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":948506,"spinlock_wait_cycles":13094144,"thread_start_us":47102,"threads_started":1,"wal-append.queue_time_us":101354}
I20260709 13:04:29.117197 15080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:29.159478 15080 raft_consensus.cc:606] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.159641 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.112061 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.161737 15162 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.160637 15154 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.117769 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.128334 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.109614 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:29.105758 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 363344us
W20260709 13:04:29.133420 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 270348us
W20260709 13:04:29.132716 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 260205us
I20260709 13:04:29.166226 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.177405 15081 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:29.131947 19526 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.112550 19416 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.185472 19416 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.184372 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.188218 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.196506 19810 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.197181 19786 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:29.198261 19810 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.198555 19780 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:29.199152 19780 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:29.200814 19780 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.049590 19756 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.203068 19780 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.203590 15156 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:29.066429 19839 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.208979 19786 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:29.229712 15081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:29.242458 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.243026 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1133) took 1105 ms. Trace:
I20260709 13:04:29.244397 15841 rpcz_store.cc:276] 0709 13:04:28.137025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.138981 (+  1956us) service_pool.cc:224] Handling call
0709 13:04:29.242562 (+1103581us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400852ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:29.242772 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500708608}
I20260709 13:04:29.243753 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.255169 19452 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.255952 19452 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.256659 15163 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:29.260704 19599 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:29.261618 19599 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.267261 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:29.240298 15813 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.108s (SENT)
I20260709 13:04:29.273820 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:29.273917 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.278541 19876 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:29.282073 19420 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:29.288938 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1139) took 1140 ms. Trace:
W20260709 13:04:29.289202 15126 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.614s (SENT)
I20260709 13:04:29.289502 15842 rpcz_store.cc:276] 0709 13:04:28.148016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.148433 (+   417us) service_pool.cc:224] Handling call
0709 13:04:29.288503 (+1140070us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54001e0fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:29.288715 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799796096}
I20260709 13:04:29.289968 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.293054 19416 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.293943 19416 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:29.299932 15237 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:29.302435 19476 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.305357 19703 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.256385 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1119) took 1513 ms. Trace:
I20260709 13:04:29.306107 15160 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:29.309223 19454 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260709 13:04:29.311933 19843 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:29.312220 15154 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:29.317435 19756 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.318915 19578 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.322974 19812 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:29.356678 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.358071 19578 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:29.325776 19599 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.341451 15162 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:29.360657 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:29.344344 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2250) took 1503 ms. Trace:
I20260709 13:04:29.361547 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:29.361984 15163 rpcz_store.cc:276] 0709 13:04:27.840730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.840896 (+   166us) service_pool.cc:224] Handling call
0709 13:04:29.344043 (+1503147us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540013ecd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:29.344254 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2631484800}
I20260709 13:04:29.346201 19452 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.346947 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.364352 19812 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:29.354653 15159 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:04:29.349854 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.363168 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:29.403685 15085 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:29.409519 19454 tablet_bootstrap.cc:492] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Bootstrap complete.
I20260709 13:04:29.410858 19454 ts_tablet_manager.cc:1403] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent bootstrapping tablet: real 2.264s	user 0.051s	sys 0.074s
I20260709 13:04:29.366906 19812 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:29.335314 15084 rpcz_store.cc:276] 0709 13:04:27.742414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.746036 (+  3622us) service_pool.cc:224] Handling call
0709 13:04:27.746323 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.813246 (+1066923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.813279 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.813779 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.125621 (+311842us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:29.126418 (+   797us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.126793 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.255539 (+128746us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400190354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:29.255612 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.255683 (+    71us) raft_consensus.cc:1685] finished
0709 13:04:29.255859 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.256139 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13384,"prepare.queue_time_us":19114,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2217,"raft.queue_time_us":47416,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":2458487296,"thread_start_us":311057,"threads_started":1,"wal-append.queue_time_us":245075}
I20260709 13:04:29.417495 19812 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:29.439836 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.449110 19780 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:29.458032 15846 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:29.458639 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.462074 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:29.472731 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47053.5us
I20260709 13:04:29.475958 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.485278 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.464951 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.506053 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.508143 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.510771 15842 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:04:29.426834 19843 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.545130 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.545444 15085 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:29.546695 15848 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:29.552965 15086 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:29.553436 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:29.554224 15840 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:04:29.554127 15051 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.911s (SENT)
I20260709 13:04:29.554978 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.556025 19786 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:29.561245 19769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.013s	sys 0.049s
W20260709 13:04:29.561689 19769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.014s	sys 0.049s
W20260709 13:04:29.564507 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43882.9us
I20260709 13:04:29.571261 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.572539 15083 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:29.571584 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:29.573889 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 145692us
I20260709 13:04:29.574774 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:29.575325 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:29.576299 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.577431 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.577878 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:29.578297 15163 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:29.578874 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.585027 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.585284 15159 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:29.606515 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.609607 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.609849 19756 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:29.610378 19786 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:29.610968 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.610417 19756 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:29.612221 19756 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.613232 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.613327 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:29.613969 15156 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.617349 19497 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:29.617763 15160 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:29.621601 15157 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:29.622954 19599 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:29.623852 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:29.623768 19599 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.624527 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.625911 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.626752 15085 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:29.626878 19599 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.626132 15159 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:04:29.627681 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.628592 15085 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:29.629132 19471 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:29.608659 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:29.602124 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.629248 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.578442 19843 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.642638 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.633965 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.637144 19780 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:29.645951 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:29.567648 19578 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:29.637118 19471 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:29.655501 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.650537 15839 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:29.656438 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47897.9us
I20260709 13:04:29.659394 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.662388 19471 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.664201 19780 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:29.660420 15083 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:04:29.669322 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.675616 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.676438 15086 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:29.677176 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 102540us
I20260709 13:04:29.675765 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.650156 19703 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:29.681999 19780 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.686954 15162 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:29.687266 19454 raft_consensus.cc:359] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 NON_PARTICIPANT]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:29.688570 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.693799 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.696990 19876 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:29.697949 19876 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:29.705149 19598 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:29.706666 15843 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:29.712735 15086 raft_consensus.cc:2410] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:29.660980 19578 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.719446 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.723039 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.668653 19756 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.698347 19876 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.705698 19598 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:29.719056 19497 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:29.690487 19818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:56640 (request call id 17) took 31104 ms. Trace:
I20260709 13:04:29.712809 19454 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Initialized, Role: LEARNER
I20260709 13:04:29.736722 15082 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:29.744745 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:29.748543 19595 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:29.749678 19559 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.750536 19599 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:29.751238 19889 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:04:29.752923 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:29.759476 19876 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.755627 15154 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:29.732967 19743 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:04:29.734411 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:29.733570 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.715163 19786 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:29.754951 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 109005us
I20260709 13:04:29.763664 19497 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.777052 15080 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:29.763720 19812 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.797649 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.798117 19664 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:29.798501 15087 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:29.798135 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45351.5us
I20260709 13:04:29.799397 19812 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.791003 19497 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.801291 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.801530 19670 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:29.802467 19843 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.802621 19769 raft_consensus.cc:2764] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:29.803165 19843 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:29.800683 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81170.7us
I20260709 13:04:29.813022 19782 raft_consensus.cc:993] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:04:29.813107 19850 raft_consensus.cc:993] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:04:29.813707 19871 raft_consensus.cc:1081] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:04:29.817148 19772 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:29.817786 19772 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:29.818192 19772 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:29.826750 19559 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.829183 19598 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.763664 19818 rpcz_store.cc:276] 0709 13:03:58.396294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.048222 (+1651928us) service_pool.cc:224] Handling call
0709 13:04:00.225812 (+177590us) log.cc:844] Serialized 513 byte log entry
0709 13:04:29.495462 (+29269650us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19052,"raft.run_cpu_time_us":13674,"raft.run_wall_time_us":299976,"thread_start_us":217935,"threads_started":1}
I20260709 13:04:29.836691 19735 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:29.801640 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 141078us
W20260709 13:04:29.854549 19859 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.126s	user 0.000s	sys 0.004s
I20260709 13:04:29.854775 19703 raft_consensus.cc:1081] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:04:29.855000 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.857707 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.785660 19786 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:29.860963 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 138114us
W20260709 13:04:29.861852 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 61008.1us
I20260709 13:04:29.865183 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.865743 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.860823 19735 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:29.877336 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 116530us
W20260709 13:04:29.878427 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 87651.2us
I20260709 13:04:29.875174 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.873253 15088 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
W20260709 13:04:29.857733 19886 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.176s	user 0.003s	sys 0.000s
I20260709 13:04:29.876921 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.778385 19599 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.881986 15082 raft_consensus.cc:2410] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:29.790725 19476 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:04:29.885329 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:29.889746 19420 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:29.890535 19420 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:29.879511 15080 raft_consensus.cc:2376] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11. Re-sending same reply.
I20260709 13:04:29.897542 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.896433 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.874563 19786 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:29.875365 19769 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:29.773972 19595 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.871263 19743 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:29.880481 19598 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.876168 19578 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.930787 19578 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:29.931429 19578 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:29.910952 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:29.914245 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:29.933949 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:29.934309 19578 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:29.916811 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:29.921953 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:29.921723 19743 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.922477 19735 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.925298 19756 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.925578 15082 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:29.927835 19595 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:29.907153 19769 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:29.929697 19843 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.952695 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.954360 19769 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.954737 19596 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:29.955298 19596 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:29.955711 19471 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.957263 19878 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:29.958201 15154 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:29.958511 19901 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:04:29.956441 19596 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:29.960173 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:29.961129 19735 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.961524 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:29.967197 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 74477.1us
I20260709 13:04:29.867233 19454 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:29.966495 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.999487 15156 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:30.002578 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.003790 15162 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:30.004314 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:30.007552 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:30.009268 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:30.009438 19599 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:30.010650 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.013244 19559 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:29.991204 19595 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:30.024829 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.029558 15082 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:30.030192 19454 ts_tablet_manager.cc:1434] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent starting tablet: real 0.619s	user 0.008s	sys 0.007s
W20260709 13:04:30.030548 19454 ts_tablet_manager.cc:1467] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Tablet startup took 619ms
I20260709 13:04:30.030768 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:29.992548 19743 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:29.950133 19420 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.038686 19471 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:30.030939 19454 ts_tablet_manager.cc:1469] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Trace:
0709 13:04:18.759356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.760079 (+   723us) service_pool.cc:224] Handling call
0709 13:04:19.233905 (+473826us) ts_tablet_manager.cc:934] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Initiating tablet copy from peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
0709 13:04:21.095894 (+1861989us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:04:21.374878 (+278984us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:21.397987 (+ 23109us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:04:22.689917 (+1291930us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:04:22.690358 (+   441us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:22.690795 (+   437us) ts_tablet_manager.cc:989] Replica registered.
0709 13:04:22.690892 (+    97us) inbound_call.cc:177] Queueing success response
0709 13:04:22.844230 (+153338us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4c003a1780. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3dcb0a20 00007f0d3dcc1a29 00007f0d3dc665e6 00007f0d3dc7bb13 00007f0d3dc7b945 00007f0d3dcee7ad 00007f0d449c2198 00007f0d449c2137 00007f0d449c209c 00007f0d449c2038 00007f0d449c1300
0709 13:04:27.146929 (+4302699us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:04:29.411418 (+2264489us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:04:29.521883 (+110465us) tablet_replica.cc:261] Starting instrumentation
0709 13:04:29.684992 (+163109us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":75585,"raft.queue_time_us":3034,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9937,"spinlock_wait_cycles":11374976,"tablet-copy.queue_time_us":79577,"tablet-download-pool.queue_time_us":3115,"tablet-download-pool.run_cpu_time_us":1419,"tablet-download-pool.run_wall_time_us":3468741,"thread_start_us":458766,"threads_started":2}
I20260709 13:04:30.043762 15080 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:30.044238 19559 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:30.045805 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 138285us
I20260709 13:04:30.026347 19595 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.058609 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.047248 19513 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:30.064028 19659 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:04:30.068861 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:30.069566 15843 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:30.070293 19659 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:30.070648 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:30.071410 15155 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:30.077059 19599 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:30.077863 19599 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.080314 19599 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.070720 19659 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:30.083072 19578 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:30.083923 19578 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:30.083491 19659 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.084259 19578 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:30.084936 19659 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:30.085196 19578 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.086577 19578 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:30.086504 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 92007.4us
I20260709 13:04:30.089320 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:30.093228 19898 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
I20260709 13:04:30.094960 19773 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:30.095364 19773 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:30.096306 19773 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.050382 19595 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.108696 15081 raft_consensus.cc:606] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:30.076545 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:30.061060 19559 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.119388 19773 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:30.098286 19513 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.129335 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.129335 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:30.143738 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:30.143865 15087 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:30.108718 19743 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.106923 19599 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.154601 15848 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:30.145577 19513 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.145829 19559 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.164963 19476 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:30.169560 19810 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.174962 19850 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.160594 19599 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.172784 19743 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.185708 19476 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.244140 15841 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:04:30.241946 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 332341us
W20260709 13:04:30.251463 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 240910us
I20260709 13:04:30.227380 19878 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:30.254690 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 372777us
W20260709 13:04:30.259178 19896 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.326s	user 0.003s	sys 0.000s
I20260709 13:04:30.260159 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.261066 15084 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:30.265625 19903 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.220s	user 0.000s	sys 0.003s
I20260709 13:04:30.266405 19743 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:30.268556 19476 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.270965 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.278453 19599 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.279156 19599 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:30.279820 19599 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:30.281250 19902 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:04:30.283331 19559 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.284113 19559 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:30.230418 19871 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.284541 19559 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:30.255904 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.304854 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.308884 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34666 (request call id 40) took 1116 ms. Trace:
I20260709 13:04:30.309500 15081 rpcz_store.cc:276] 0709 13:04:29.192311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.216445 (+ 24134us) service_pool.cc:224] Handling call
0709 13:04:30.308226 (+1091781us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:30.308315 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018498816}
W20260709 13:04:30.317137 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 120895us
W20260709 13:04:30.311981 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.043s (SENT) [suppressed 28 similar messages]
W20260709 13:04:30.323381 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 246904us
I20260709 13:04:30.305018 19756 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:30.305655 19559 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.309413 19918 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.346969 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.350936 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:30.249969 19596 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:04:30.363307 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.364722 19769 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.367671 19871 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.370239 19871 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.374207 19769 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.373950 14982 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 5 to 8, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:30.378639 19843 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:30.380534 19908 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:30.381451 19907 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:30.382030 19907 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.384191 19907 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:30.358623 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:30.390153 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 299098us
I20260709 13:04:30.376466 14982 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 7 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.252902 19850 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:30.426376 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 107737us
W20260709 13:04:30.423307 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 254538us
W20260709 13:04:30.425223 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 118326us
I20260709 13:04:30.390890 19756 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:30.475811 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 205046us
I20260709 13:04:30.384706 19907 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:30.516289 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:30.517308 15156 raft_consensus.cc:2410] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:04:30.516856 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 204886us
W20260709 13:04:30.517748 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 154515us
I20260709 13:04:30.391146 19786 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.523317 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:30.510456 19780 raft_consensus.cc:2764] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:30.521661 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:30.529980 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.534602 15159 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:30.514194 19907 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:30.541447 15161 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:30.545658 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.517666 19871 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.540037 19788 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.549376 19871 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:30.550897 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.551944 19871 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.552098 19735 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.552693 19735 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.554774 19735 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:30.554916 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.555274 19735 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:30.556188 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.557826 19743 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:30.558437 19743 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.558909 19513 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.560499 19743 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.567800 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:30.568529 15156 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:30.517268 19452 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:04:30.537488 19772 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.000s
I20260709 13:04:30.561015 19813 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:30.510041 19810 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.561265 15097 tablet_copy_service.cc:342] P 12daa67457954a0cab27e32bf85fa115: Request end of tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ff3342a5b2ad49e2b252d7cd47b9851d received from {username='slave'} at 127.0.0.1:56652
I20260709 13:04:30.584223 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:30.425658 19918 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:30.541548 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 227875us
I20260709 13:04:30.569485 15086 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:30.585052 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49289.1us
I20260709 13:04:30.536103 19420 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.601923 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.601658 15848 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEARNER]: Advancing to term 10
W20260709 13:04:30.583951 19497 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.779s	user 0.017s	sys 0.000s
W20260709 13:04:30.590804 19659 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:04:30.626389 15839 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:30.613945 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.601621 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:30.580111 19876 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.737s	user 0.014s	sys 0.000s
I20260709 13:04:30.618732 15081 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:30.602195 14982 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.600965 15097 tablet_copy_service.cc:434] P 12daa67457954a0cab27e32bf85fa115: ending tablet copy session 3c28be6f86ea4fb6b79a47d3999f5a82-ff3342a5b2ad49e2b252d7cd47b9851d on tablet ff3342a5b2ad49e2b252d7cd47b9851d with peer 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.584168 19813 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:30.628863 15085 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:04:30.667790 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:30.527915 19878 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:30.667465 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 116849us
W20260709 13:04:30.677004 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124639us
I20260709 13:04:30.692270 15162 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:30.618707 19703 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.594565 19780 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:04:30.632447 19599 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:30.584004 19772 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.668682 19813 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:30.703406 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.625123 19871 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.622074 19773 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 13:04:30.721339 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 191433us
W20260709 13:04:30.708093 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 184826us
I20260709 13:04:30.718340 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.623055 19420 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.637398 19876 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.664611 19930 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:30.675261 19659 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:04:30.705060 19907 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.726116 19756 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.737809 19929 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:30.760998 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.781536 19929 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:30.781003 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:30.754530 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.781747 15160 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:04:30.742749 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 67213.6us
I20260709 13:04:30.725435 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:30.759284 15848 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEARNER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
W20260709 13:04:30.799368 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68748.3us
W20260709 13:04:30.749140 19595 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.002s
W20260709 13:04:30.785964 19578 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.001s
I20260709 13:04:30.802686 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:30.757769 19922 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:30.777047 19598 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:30.807021 15841 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:30.741820 19786 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.743592 19924 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.758138 19513 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.756475 19918 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.807713 19452 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:04:30.825753 15163 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:30.825958 19452 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:30.826320 19452 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:30.827271 19452 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.827553 19813 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.808779 19931 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.828627 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103291us
I20260709 13:04:30.799914 19930 consensus_queue.cc:1243] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:30.799831 19818 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.809898 19933 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:30.754776 19871 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.807524 19595 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.811308 19937 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.809469 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55065.9us
I20260709 13:04:30.737996 19923 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:30.738155 19926 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.731042 19703 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.814263 19939 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:30.814560 19928 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:30.815272 19929 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.815532 19735 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:30.769287 19599 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.816191 19935 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:30.758697 19850 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.817098 19942 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:30.819571 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:30.770852 19915 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.820701 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:30.773697 19559 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.851643 19786 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:04:30.822358 19420 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.822481 19876 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:30.823908 19598 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:30.728982 19780 consensus_queue.cc:1243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:30.759294 19773 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:30.827864 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:30.922771 15158 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.809044 19772 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:30.809576 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.844789 19476 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:30.846771 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.848454 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:30.925321 15080 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.850016 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:30.851560 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:30.860379 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:30.729224 19812 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:04:30.858412 19818 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.634568 19497 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.842059 19924 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:30.883713 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 354536us
I20260709 13:04:30.939136 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.885771 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:30.891054 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:30.894550 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 132897us
I20260709 13:04:30.895241 19942 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:30.861797 19938 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:30.944412 15158 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:30.948578 15843 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:30.922166 19850 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:30.880265 19928 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.859990 19931 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.838296 19452 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.927155 19735 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.927706 19598 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.928668 19876 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:30.933483 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124026us
I20260709 13:04:30.933938 19843 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:30.916432 19769 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:30.913780 19578 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:30.956502 19599 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:30.979096 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93456.3us
I20260709 13:04:30.947033 15083 raft_consensus.cc:2410] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:04:30.821269 19907 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:30.955747 15841 pending_rounds.cc:85] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 7
I20260709 13:04:30.862056 19937 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.956736 15085 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:30.865998 19923 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:30.885361 19926 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.948094 19942 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.964475 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:30.967689 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:30.956032 19420 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:30.956347 19928 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.951874 19850 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:30.940804 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:30.947477 19924 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.981979 19813 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.970937 19578 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:30.971135 19769 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:30.983908 19743 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:30.984705 19476 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:30.985085 19598 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:30.985846 14982 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.986514 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:30.986781 19818 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.987093 19927 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:30.988091 19871 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:30.989388 19921 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:31.079072 19945 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:31.079162 19921 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:31.080607 19921 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.082033 19921 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:30.963479 19735 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:31.004180 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.924865 19915 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.005366 15080 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:31.005947 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 199900us
I20260709 13:04:30.963459 19876 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:30.993110 15841 pending_rounds.cc:107] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:30.982642 19931 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:31.043721 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83463.5us
I20260709 13:04:30.958034 19938 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.089998 19876 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.090317 15841 raft_consensus.cc:2889] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.091806 15839 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.059115 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.093487 19497 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:31.047848 19914 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.095331 19938 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:31.060665 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 93060.4us
I20260709 13:04:31.095571 19578 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.116767 19871 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.117462 19871 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.047118 19599 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:31.120301 19599 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.120726 19599 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:31.085887 19735 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.064999 19935 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:31.065459 19927 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:31.067576 19559 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.068807 19930 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:31.069367 19934 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:31.056447 19743 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:31.076129 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.134397 15085 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:31.078100 19923 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.091224 19876 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:31.061349 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 120630us
W20260709 13:04:31.063067 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 76621.5us
I20260709 13:04:31.046916 19812 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:31.048955 19924 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:31.098634 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.099542 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.099864 19937 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.100463 19850 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.101714 19703 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.102357 19769 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.104521 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.105180 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.105468 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.106184 15840 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:31.106134 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.106931 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.180140 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.180366 15842 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:31.107995 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.179257 15845 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:31.181428 15843 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.062062 19595 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:31.111896 19947 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:04:31.114108 14982 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 7 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:31.187738 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 81750.9us
I20260709 13:04:31.188241 14982 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.189579 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:31.190284 19559 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:31.190214 14982 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:04:31.191298 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.191748 14982 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.116044 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.193830 15839 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.194586 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.195740 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.125717 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:31.117434 19578 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.140583 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:31.140605 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.142992 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.123083 19935 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:31.148962 15087 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:31.192600 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.197089 15083 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:31.201220 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60703.9us
I20260709 13:04:31.202123 15848 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.185526 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:31.204190 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.163075 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.130636 19930 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:31.134052 19934 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:31.117519 19945 consensus_queue.cc:1175] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 9 index: 5 } }
W20260709 13:04:31.176476 15201 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.701s (SENT)
I20260709 13:04:31.176935 15157 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:31.138489 19927 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:31.140996 19926 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.185526 19915 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.185458 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:31.158874 14981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 368) took 12975 ms (client timeout 15000 ms). Trace:
I20260709 13:04:31.196801 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.241792 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.259492 15848 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:31.240299 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.207190 19935 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.269927 14982 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 6 to 7, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 7 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.260562 15839 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
W20260709 13:04:31.271942 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 172469us
I20260709 13:04:31.272830 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.154071 19818 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:04:31.253633 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.255259 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.273875 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.279281 19934 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.280519 19934 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:31.281445 19927 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.283286 15083 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:31.285310 15159 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:31.285063 14982 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 5 to 7, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 7 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.286787 14982 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 4 to 8, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.288371 14982 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.254349 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:31.257402 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80962.9us
W20260709 13:04:31.303772 15201 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.716s (SENT)
I20260709 13:04:31.303834 19926 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:31.304281 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.304414 19926 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:31.184417 19850 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.305352 19926 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.307852 19559 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.258688 15842 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.309024 15847 raft_consensus.cc:2410] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:31.316454 15842 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.239969 19786 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:31.183835 19703 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.154392 19812 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.203776 15843 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:31.200711 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:31.292511 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 103023us
I20260709 13:04:31.255808 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.301831 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.257237 19813 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:31.319015 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.251942 19788 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:31.324951 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 145996us
I20260709 13:04:31.325562 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.361881 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:31.336755 15201 leader_election.cc:336] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.731s (SENT)
W20260709 13:04:31.354213 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 153587us
I20260709 13:04:31.355332 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.355839 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.375388 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:31.375905 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 103158us
I20260709 13:04:31.375191 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.380157 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:31.382681 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.373366 19818 consensus_queue.cc:1243] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:31.381047 15841 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:31.385615 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.386847 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.387481 15841 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.376452 14982 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 3 to 6, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 6 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.375134 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.360056 19598 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:31.359902 19703 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.389562 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.390797 15234 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:31.355971 19595 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:31.356449 19813 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.356487 19780 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.393357 15840 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.392031 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.385672 19559 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:04:31.396497 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:31.405289 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.413405 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.414441 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.415397 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.416388 19703 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.417287 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.431115 15232 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:31.392738 19930 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.418144 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.419028 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.419574 19812 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.419881 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.265444 19743 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.423632 19928 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.355054 19786 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.428898 15230 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:31.444905 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 108187us
I20260709 13:04:31.393024 19818 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:31.447482 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.449088 15235 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:31.448925 14982 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.457387 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.457214 14982 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 5 to 8, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 8 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:31.458387 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.458575 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.459143 15235 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:31.458904 19942 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.460105 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.434468 19813 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:31.442356 19953 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:04:31.462288 19928 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.442688 19780 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.451395 19818 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:31.467008 19780 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.467231 19818 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.467743 19871 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:31.468436 19818 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:31.469442 19786 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.469656 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.470628 15233 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:04:31.470755 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.446210 15229 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:31.471879 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:31.443863 19950 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.187s	user 0.002s	sys 0.000s
I20260709 13:04:31.454789 15236 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:31.472349 15846 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:31.472615 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.471879 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.493074 15839 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.494355 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.495111 15839 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:31.496225 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.473436 15843 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.433961 19598 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:31.458999 15228 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:31.478978 15848 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:31.498704 15841 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
W20260709 13:04:31.499293 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 61366.4us
I20260709 13:04:31.500419 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:31.504182 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.506793 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.455452 19930 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.516206 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.517428 15841 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:31.519268 15236 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 9: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:31.519963 15841 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:31.520207 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.523718 15843 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:31.524829 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.525619 15235 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:31.529959 15848 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:31.531239 15839 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:31.507845 19926 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:31.533213 15236 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:31.537441 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 150249us
I20260709 13:04:31.491847 19476 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:04:31.540262 15229 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:31.258646 14981 rpcz_store.cc:269] 0709 13:04:18.183423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.183608 (+   185us) service_pool.cc:224] Handling call
0709 13:04:31.158549 (+12974941us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:26.826772 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:27.054727 (+227955us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:27.064396 (+  9669us) write_op.cc:435] Acquired schema lock
0709 13:04:27.071715 (+  7319us) tablet.cc:662] Decoding operations
0709 13:04:27.206854 (+135139us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:27.206941 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:27.206981 (+    40us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:04:27.229795 (+ 22814us) tablet.cc:701] Row locks acquired
0709 13:04:27.229826 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:04:27.230079 (+   253us) write_op.cc:270] Start()
0709 13:04:27.230230 (+   151us) write_op.cc:276] Timestamp: P: 1783602267229977 usec, L: 0
0709 13:04:27.230263 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:04:27.399166 (+168903us) log.cc:844] Serialized 8203 byte log entry
0709 13:04:28.094019 (+694853us) op_driver.cc:464] REPLICATION: finished
0709 13:04:28.575004 (+480985us) write_op.cc:301] APPLY: starting
0709 13:04:28.576235 (+  1231us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:29.064446 (+488211us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:29.064752 (+   306us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:29.835489 (+770737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:29.835931 (+   442us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:04:29.836004 (+    73us) write_op.cc:312] APPLY: finished
0709 13:04:29.836723 (+   719us) log.cc:844] Serialized 180 byte log entry
0709 13:04:30.243372 (+406649us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:30.259368 (+ 15996us) write_op.cc:454] Released schema lock
0709 13:04:30.259476 (+   108us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:30.264379 (+  4903us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:30.264411 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":483982,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":464763,"apply.run_cpu_time_us":115780,"apply.run_wall_time_us":1696046,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87847,"num_ops":20,"prepare.queue_time_us":819098,"prepare.run_cpu_time_us":39748,"prepare.run_wall_time_us":875661,"raft.queue_time_us":260238,"raft.run_cpu_time_us":42789,"raft.run_wall_time_us":463948,"replication_time_us":806027,"thread_start_us":1707078,"threads_started":5,"wal-append.queue_time_us":763569,"wal-append.run_cpu_time_us":23759,"wal-append.run_wall_time_us":2232012}]]}
I20260709 13:04:31.529165 19942 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:31.538113 19926 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.544247 19664 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.569229 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:31.573520 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:31.575049 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74725.3us
I20260709 13:04:31.575886 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:31.577200 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.580150 19929 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:31.580497 19514 raft_consensus.cc:2764] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:04:31.580660 19929 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:31.581139 19929 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:31.562136 19928 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.563127 19926 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.589740 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.544697 19924 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.600116 15229 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.605125 19939 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:31.626956 15230 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:04:31.629016 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55402.8us
I20260709 13:04:31.630168 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.631537 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:31.585271 19664 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.635857 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:31.645205 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.622741 19471 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.657197 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.663615 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.663414 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.667544 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.629454 19923 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.693943 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 363905us
W20260709 13:04:31.696470 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.697137 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.605697 19928 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.691370 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:31.699038 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.668496 19939 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:31.703403 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.705720 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.671775 19497 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:31.604903 19598 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.708278 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:31.713174 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 123646us
I20260709 13:04:31.715729 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.717433 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.717762 15085 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:31.718330 15082 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:31.720062 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:31.721094 15087 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:04:31.721932 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.703917 19813 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.674113 19471 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.726038 15084 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:31.706024 19939 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:31.736732 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:31.740128 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72694.5us
I20260709 13:04:31.740563 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.742960 15080 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:31.745353 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.709342 19497 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:31.749239 19578 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:31.749620 19497 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.750571 19578 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.752986 19659 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.745748 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:31.745926 15234 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:31.745368 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:31.724660 19956 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:04:31.702373 19923 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:31.761010 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:31.743799 19876 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
W20260709 13:04:31.758133 19921 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.002s
I20260709 13:04:31.779043 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:31.790607 15845 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:31.746330 19939 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:31.785984 15234 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:31.774696 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.796947 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:31.793288 15088 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:31.798336 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.783151 19497 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.799122 15847 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.801144 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:31.803823 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.805320 19955 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:04:31.806231 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:31.807531 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.801177 19497 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.808173 15233 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:31.809669 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:31.809881 19928 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.811731 15155 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:31.813408 19471 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:31.815713 19937 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:31.816252 19937 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:31.824321 19961 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
W20260709 13:04:31.825654 19970 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:31.830175 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.831106 15847 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.824512 19937 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.833053 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:31.833797 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.837776 15228 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:31.838940 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.843274 15229 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:31.844913 15842 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:31.845002 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 99691.5us
I20260709 13:04:31.845469 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.845854 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:31.785962 19964 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:04:31.846720 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:31.778609 19963 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:04:31.854885 15229 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:04:31.794625 19596 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:31.846318 15234 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:31.847620 15848 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:31.849458 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.746912 19769 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:04:31.768467 19942 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:31.856870 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.862540 15845 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:31.768908 19788 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:31.861227 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:31.858850 15841 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:31.859226 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.863906 15155 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.864302 15163 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.860088 15233 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:31.849838 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:31.866295 15085 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.847087 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.782651 19813 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.811720 19924 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:31.877130 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:31.855479 15229 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:04:31.878937 15229 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:04:31.879314 15229 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.860504 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:31.870705 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:31.872291 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:31.880160 15159 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.855317 19497 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.894604 19497 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.897119 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.884665 19927 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:31.907290 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.894891 15234 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:31.905808 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.908738 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.906327 15158 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:31.909891 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.905237 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.910588 15162 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:31.909891 15157 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:31.911024 15154 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:31.911863 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.912245 19497 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:31.914170 19978 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.914017 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.912756 19907 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.917341 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.918932 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:31.923820 19928 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:31.913203 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.928864 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:31.929898 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:31.931098 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:31.914065 15229 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:04:31.939592 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:31.846037 19968 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:31.855902 19514 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.943910 15844 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:31.945043 15085 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:31.940657 19559 raft_consensus.cc:2764] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:31.873435 19976 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:31.885674 19937 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:31.943018 19876 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:31.957379 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1197) took 1027 ms. Trace:
I20260709 13:04:31.941011 19497 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.951383 19964 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:04:31.947947 19578 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:31.949491 19978 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:31.965438 15846 rpcz_store.cc:276] 0709 13:04:30.930312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.930473 (+   161us) service_pool.cc:224] Handling call
0709 13:04:30.930811 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.491717 (+560906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.952737 (+461020us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54004a2754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:31.957319 (+  4582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244826496}
W20260709 13:04:31.975960 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.948115 19975 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:31.956290 19514 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:31.967480 19926 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.001550 15156 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:32.002887 19514 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.002985 19926 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:31.965086 19876 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.005019 19876 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.006300 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:32.010679 15235 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:32.012825 15159 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.015420 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:31.991462 19595 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:32.025785 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.019801 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.035621 19769 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.036129 19769 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.036510 19769 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.018357 15232 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:31.992061 19975 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:32.039562 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.009699 19578 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:32.038344 15080 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:31.981355 19497 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.041515 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.045328 15232 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:04:32.046171 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.041592 19599 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.920s	user 0.009s	sys 0.008s
I20260709 13:04:32.049911 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:32.043678 19818 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.003s
I20260709 13:04:32.041869 19975 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:32.053751 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.055402 15084 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:32.051672 19599 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.057147 19578 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.058487 19788 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:32.058782 19599 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.059806 15231 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:32.056754 19664 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.020514 19980 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:32.073907 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.070245 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.050203 19930 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:32.076210 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 279748us
I20260709 13:04:32.079864 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.056145 19907 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:32.094463 19930 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:32.099679 15155 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:32.102648 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:32.106187 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.104144 15235 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.077948 19929 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.108242 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.091859 19514 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:32.125993 15161 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:32.087839 19813 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:32.145725 15230 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
W20260709 13:04:32.145699 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 213482us
I20260709 13:04:32.110051 19780 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.115257 19921 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:32.147990 19995 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:32.167261 15232 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:32.168345 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.169495 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.149813 19780 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.169919 15235 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.150460 19921 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.174844 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:32.177474 15126 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.859s (SENT)
I20260709 13:04:32.169138 19514 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:32.179783 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.180578 15235 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:32.171020 19995 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:32.176612 19921 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.189548 19995 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.191443 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.193140 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.194097 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.194337 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.194888 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.195223 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.195868 15229 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:32.185074 19974 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.436s	user 0.003s	sys 0.000s
I20260709 13:04:32.196892 19599 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:32.197981 19599 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.198372 19599 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:32.198740 19599 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.198932 19907 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:32.199558 19907 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:32.181823 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.173661 19780 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.202059 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.196784 19968 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:32.204950 19497 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.205045 19996 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:32.205307 19968 consensus_queue.cc:1243] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:32.205955 19968 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:32.206064 19986 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:04:32.206336 19968 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:32.207242 19968 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.207990 19929 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:32.208523 19968 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:32.178057 19930 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.212977 15228 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:04:32.210440 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.217648 15228 pending_rounds.cc:85] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 10
I20260709 13:04:32.217958 15228 pending_rounds.cc:107] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:04:32.218122 15840 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:32.218171 15228 raft_consensus.cc:2889] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.219637 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.220480 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.221602 15229 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:32.188349 19975 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.226326 19813 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:32.226544 19939 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.228138 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:32.233986 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.235920 15846 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:32.237699 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.239092 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.179381 19981 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:32.242599 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:32.198760 19595 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.246088 19595 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.249082 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:32.249215 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.250699 19924 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.251127 19924 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.251485 19924 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.251567 19769 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.251894 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.252979 19981 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.255234 19981 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.256283 19981 raft_consensus.cc:2764] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:32.200450 19907 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.259032 19907 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.263401 19975 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.267905 19818 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:32.208696 19876 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.270778 19876 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:32.229943 19939 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.271425 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.276202 19921 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.277187 19514 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:32.279903 19973 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.481s	user 0.002s	sys 0.000s
I20260709 13:04:32.228765 19780 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.282856 19780 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.244865 19497 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:32.286288 19497 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:32.286798 19497 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:32.233229 19813 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:32.243575 19786 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:32.289503 19786 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:32.289918 19786 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.291249 19818 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:32.296775 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.297257 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.299600 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.300161 19923 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:32.301608 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.302793 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.303057 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.303311 15237 raft_consensus.cc:2376] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10. Re-sending same reply.
I20260709 13:04:32.303699 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.304129 19813 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.304868 19813 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.307030 19813 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.307816 15234 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:32.309811 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.304149 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.310928 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.287933 19497 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.311717 15844 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:32.313148 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.314309 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.314533 19786 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.315361 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.315902 15228 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.304477 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.316681 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.388620 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:32.352063 15232 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.342994 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:32.352993 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.342784 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.327631 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:32.356491 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.357605 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:32.434178 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 326828us
I20260709 13:04:32.316604 19786 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:32.358043 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.346478 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.324352 15162 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:32.433727 15086 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.300163 15229 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11. Re-sending same reply.
W20260709 13:04:32.450564 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 201560us
I20260709 13:04:32.459205 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.457553 15234 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.324865 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.346190 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:32.319882 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.461036 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:32.462388 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:32.326591 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.463299 15155 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.461272 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:32.463614 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 137112us
I20260709 13:04:32.464457 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.463438 15085 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:32.464704 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:32.465157 15841 raft_consensus.cc:2368] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11.
W20260709 13:04:32.322954 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.465968 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.466537 15842 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:32.462767 15846 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:32.431207 15231 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:32.460745 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:32.466696 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.439851 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.350814 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.340437 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:32.469241 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 118524us
I20260709 13:04:32.460000 15229 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:32.375382 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:32.423705 15237 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
W20260709 13:04:32.439935 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 82410.9us
I20260709 13:04:32.440089 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.472635 15082 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:32.434106 15230 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
W20260709 13:04:32.473253 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 126912us
I20260709 13:04:32.473227 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:32.469455 15233 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.474464 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:32.467994 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.462441 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:32.345366 19769 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.478623 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.479184 19664 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:32.479391 15088 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:32.479615 19664 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.479966 19664 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.480190 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.481113 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:32.481851 15083 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:32.482898 15087 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:32.483268 19968 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:32.483857 19968 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:32.484766 19968 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.487972 19997 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:32.488963 19420 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:32.489323 15840 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:32.489461 19420 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:32.489904 15228 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:04:32.490350 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.490422 19420 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.491756 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:32.475512 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:32.488201 19664 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.354260 19935 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.398430 19497 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.497432 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.499128 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:32.500393 15841 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:32.322065 19743 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:32.523092 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47434.3us
I20260709 13:04:32.523952 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:32.524906 15235 raft_consensus.cc:1217] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[6.5-10.7]   Dedup: 6.5->[8.6-10.7]
I20260709 13:04:32.525600 15235 pending_rounds.cc:85] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
I20260709 13:04:32.525395 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.525951 15235 pending_rounds.cc:107] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 9.6
W20260709 13:04:32.502846 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 127530us
I20260709 13:04:32.526338 15235 raft_consensus.cc:2889] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.488605 19997 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:32.505844 15229 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:04:32.323449 19818 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.397017 19921 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:32.508563 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68555.3us
W20260709 13:04:32.509554 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47365.8us
I20260709 13:04:32.541311 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.542471 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.542963 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:32.413324 19907 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:04:32.362787 19595 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.356511 19996 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:32.497043 20003 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:32.440462 19980 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:32.319907 19924 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.547729 15842 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:32.548005 15229 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:04:32.568013 15229 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:32.548684 15236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:04:32.568466 15229 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.556993 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.356361 19999 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.186s
I20260709 13:04:32.522326 19786 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:32.562544 15234 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:32.519443 15155 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:32.544296 19942 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:32.317996 19599 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.500790 19876 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:32.522603 19930 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:32.536518 19995 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:32.574843 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.575245 15052 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:32.579121 15237 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:32.580651 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.552016 19596 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:32.563181 15086 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:32.606994 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.628798 15840 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:32.621740 15088 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:32.606277 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:32.555006 19773 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:32.613107 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 114072us
I20260709 13:04:32.552170 19921 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:32.582736 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.614032 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:32.555658 19958 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:32.552482 19514 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:32.616658 15087 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:32.558009 19813 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:32.653491 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:32.552234 19871 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:04:32.671286 15236 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:32.671859 15236 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.639262 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.586277 20003 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:32.632867 15229 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:32.655164 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 97957.2us
I20260709 13:04:32.659773 15085 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:32.569368 19598 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:04:32.558096 19923 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:04:32.666101 15088 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:32.684453 15229 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:32.684913 15085 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:32.606052 19818 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:32.676242 15234 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:32.661175 15847 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:32.621327 19995 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:32.636413 15237 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:32.623556 19930 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:32.571853 19996 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.652614 19921 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:32.709064 15237 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:04:32.709626 15237 pending_rounds.cc:85] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:04:32.709932 15237 pending_rounds.cc:107] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:04:32.710234 15237 raft_consensus.cc:2889] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.712216 15237 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:32.630749 19934 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:04:32.730319 15085 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:32.732059 15085 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 9
I20260709 13:04:32.732342 15085 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:32.728070 15232 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:32.732610 15085 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.734575 15127 leader_election.cc:400] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11.
I20260709 13:04:32.734905 15127 leader_election.cc:403] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:04:32.698443 19995 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:32.738754 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 99586.2us
I20260709 13:04:32.734336 19452 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:32.736405 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 179537us
I20260709 13:04:32.739712 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.697099 19930 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.740654 15841 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:04:32.742558 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:32.737454 19513 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:32.684455 19999 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:32.703665 19921 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:32.678846 19514 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.749605 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.686877 19773 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:32.744071 19513 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:32.784135 15845 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:04:32.780460 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.781859 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.727556 20000 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:32.738608 19818 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.783074 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:32.787858 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.790725 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:32.747440 19876 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.730547 19928 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:32.730926 19927 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:04:32.802587 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.698094 19997 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.788107 20000 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:32.789744 19813 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:32.746886 19452 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.790078 19818 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:32.855078 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:32.834093 19786 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:32.832904 19664 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:32.833040 19999 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11.
W20260709 13:04:32.873083 20007 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:04:32.835301 19942 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:32.892302 19942 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.876436 19818 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.875128 19924 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.896322 19924 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:32.898115 19452 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:32.898813 19452 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:32.899209 19452 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:32.900223 19452 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:32.905383 20004 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 13:04:32.907241 20005 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:04:32.854007 20000 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.879807 19971 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:32.909797 19971 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:32.910534 19971 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:32.832880 19915 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:32.879313 19928 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:04:32.913673 19927 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:32.914042 19915 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.914464 19599 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.866128 19769 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.914562 19915 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.873194 20014 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.916040 20014 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.916385 20014 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:32.884873 20010 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:32.872879 20012 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.882512 19786 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.869930 19513 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.807616 19876 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.914055 19927 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:32.921288 19927 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.911401 19995 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.922597 19927 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.925619 19996 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:32.926383 19996 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:32.926893 19996 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:32.927974 19996 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.934469 15233 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:32.799736 20003 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:32.943542 19915 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.945477 15236 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:32.949818 19876 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.951370 19514 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:32.943403 15846 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:32.957999 15848 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:32.968361 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 393173us
I20260709 13:04:32.970700 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:32.976894 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.912428 19971 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:04:32.878325 19596 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:33.001996 19995 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.002369 19788 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:33.002933 19788 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:33.005096 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.005771 15085 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:33.006083 15085 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.003973 19788 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.023753 19921 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:33.027055 19935 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.000s
W20260709 13:04:33.027428 20006 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.407s	user 0.000s	sys 0.010s
I20260709 13:04:33.028098 19598 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.054373 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.057785 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 315287us
I20260709 13:04:33.006799 15085 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.079133 15085 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:33.080003 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.080255 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:33.094336 20009 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.358s	user 0.009s	sys 0.009s
I20260709 13:04:33.094902 19513 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:33.095748 19497 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:33.098453 20017 raft_consensus.cc:1064] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
W20260709 13:04:33.098933 20017 raft_consensus.cc:1068] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Unable to promote non-voter 3c28be6f86ea4fb6b79a47d3999f5a82: Service unavailable: leader transfer in progress
I20260709 13:04:33.099261 19514 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:33.100473 19497 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:33.115386 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.116364 15084 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:33.122252 15085 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:33.123836 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2231) took 1110 ms. Trace:
I20260709 13:04:33.124361 15235 rpcz_store.cc:276] 0709 13:04:31.997445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.474623 (+477178us) service_pool.cc:224] Handling call
0709 13:04:32.524627 (+ 50004us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:32.557742 (+ 33115us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:32.557773 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:32.576008 (+ 18235us) log.cc:844] Serialized 52 byte log entry
0709 13:04:32.984494 (+408486us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:32.984559 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.984655 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.107918 (+123263us) raft_consensus.cc:1685] finished
0709 13:04:33.107938 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.108087 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1565,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":21101,"spinlock_wait_cycles":2816,"thread_start_us":313741,"threads_started":1,"wal-append.queue_time_us":451504}
W20260709 13:04:33.128185 15154 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.012s	sys 0.005s
W20260709 13:04:33.130815 15154 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.005s
I20260709 13:04:33.078959 20020 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:33.100879 19497 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:33.143450 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.002s	sys 0.021s
W20260709 13:04:33.144791 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.002s	sys 0.021s
W20260709 13:04:33.146663 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 602897us
I20260709 13:04:33.152438 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:33.153800 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 497534us
I20260709 13:04:33.101243 19598 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.159220 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.125870 19514 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:04:33.157596 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 417929us
I20260709 13:04:33.161880 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.163473 15161 raft_consensus.cc:2368] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:33.163757 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:33.165912 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 285959us
I20260709 13:04:33.166373 15156 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.159790 15157 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11. Re-sending same reply.
I20260709 13:04:33.167505 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:33.175019 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.175504 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.177572 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.181766 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.184654 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.185737 15154 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:33.185485 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:33.186538 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:33.185451 15163 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:33.154968 19813 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.190793 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:33.140982 20014 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.140551 19769 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:33.195923 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.196596 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:33.166623 19958 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.228756 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.225567 15236 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.180962 19928 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:33.234050 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.184852 20021 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:33.191504 19942 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:33.235545 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43960 (request call id 2244) took 1164 ms. Trace:
W20260709 13:04:33.267536 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81045.8us
I20260709 13:04:33.271063 19921 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.272235 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.300658 15083 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:33.274044 15086 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:33.275935 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.277187 15079 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:33.201416 19743 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:04:33.309489 20011 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.237612 15054 leader_election.cc:400] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:33.310245 15054 leader_election.cc:403] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:33.310364 20019 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:33.288164 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.277848 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.280071 15232 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:33.281906 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.282109 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.282696 15158 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:33.242455 15236 rpcz_store.cc:276] 0709 13:04:32.065676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.546087 (+480411us) service_pool.cc:224] Handling call
0709 13:04:33.189253 (+643166us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9dba3d 00007f0d449abcd0 00007f0d3df30254 00007f0d3df301e2 00007f0d3df30111 00007f0d3df3008c 00007f0d3df2f342 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:33.230224 (+ 40971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235915648}
I20260709 13:04:33.151503 19596 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.296252 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:33.296324 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.322192 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.278901 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:33.311170 20019 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:33.320561 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83149.9us
I20260709 13:04:33.326354 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:33.324081 15161 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.326733 15160 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:33.195029 19915 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:04:33.126039 20012 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:33.330637 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:33.269264 19927 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.305054 20015 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 13:04:33.335357 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:33.340716 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:33.342857 15847 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:33.345348 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66526.4us
I20260709 13:04:33.228863 19930 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:33.339224 19871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.029s	sys 0.069s
W20260709 13:04:33.346869 19871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.029s	sys 0.069s
I20260709 13:04:33.193289 19497 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.347447 20026 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:33.276054 19958 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.228626 20000 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.226998 19818 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.311560 20021 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:33.335713 19915 consensus_queue.cc:1243] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:33.382390 15161 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:33.383792 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.394251 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.321363 19928 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:04:33.399675 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49293.2us
I20260709 13:04:33.397115 15155 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:33.397893 15846 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:33.398790 15156 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.402479 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.402937 20038 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.403600 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.403512 20038 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.405687 20030 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:33.406132 20030 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.406533 20030 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:33.399667 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.339601 20034 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.409611 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 72743.1us
I20260709 13:04:33.411729 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.412673 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:33.396762 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.414875 15161 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.414063 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:33.417725 15084 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:33.310118 20011 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.390349 19871 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:33.428400 19871 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.428933 19871 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.437342 15154 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.377362 20037 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:33.438426 20011 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.438684 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.438382 19871 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.439553 20037 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:33.440294 20037 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.441015 19871 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.441617 19871 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:33.443395 20021 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.378551 20026 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:33.448943 20019 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.451020 20026 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.361402 19971 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
W20260709 13:04:33.453274 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 680596us
I20260709 13:04:33.454490 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2242) took 1412 ms. Trace:
I20260709 13:04:33.456173 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:33.460013 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 264863us
I20260709 13:04:33.460319 15082 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:33.381409 19769 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:33.460856 15082 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.350595 19934 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:33.470230 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:33.470378 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.390460 19921 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:04:33.471014 15846 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:33.473567 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:33.479117 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.480438 15163 raft_consensus.cc:1217] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Deduplicated request from leader. Original: 9.10->[10.11-10.11]   Dedup: 10.11->[]
I20260709 13:04:33.481060 15163 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:04:33.459512 15231 rpcz_store.cc:276] 0709 13:04:32.042302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.509238 (+466936us) service_pool.cc:224] Handling call
0709 13:04:32.509495 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.536653 (+ 27158us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:32.570746 (+ 34093us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.746604 (+175858us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.988517 (+241913us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800432b20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:33.441535 (+453018us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b1800432b20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d438b858f 00007f0d3c8f7571 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd
0709 13:04:33.454169 (+ 12634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.454222 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:33.454270 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.454337 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.454439 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":680561,"prepare.queue_time_us":53147,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":141,"raft.queue_time_us":22789,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":8946816,"thread_start_us":102700,"threads_started":1,"wal-append.queue_time_us":485572}
I20260709 13:04:33.463586 19975 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:04:33.437347 19818 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.343743 19927 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:33.488153 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 70717.9us
I20260709 13:04:33.421283 20034 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.477216 15082 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.496491 15082 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:33.497309 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:33.361918 19997 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:33.470359 20014 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.506858 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.478467 20026 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:33.508975 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45008.4us
I20260709 13:04:33.509070 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.512607 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.513271 15160 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.514708 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.515518 15161 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.520304 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.509790 19997 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:33.400331 19813 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:33.553996 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.556059 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.557098 15086 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.504163 19968 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.520987 19923 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.556494 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.563773 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.564749 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.578626 15156 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:33.521790 20026 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.580987 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.581792 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.549518 19975 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:33.600135 15163 raft_consensus.cc:2410] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:04:33.498040 19934 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.549347 19980 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.549172 20046 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:33.614698 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.616644 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.617587 15163 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:33.593304 20026 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.559324 20059 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:33.603284 19975 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.576066 19968 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.560129 20037 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.628566 19968 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.599300 20049 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.634713 20018 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.593618 19923 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.648187 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.614328 20038 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.651805 15080 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:33.624266 19930 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.638257 19968 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.613654 19980 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.658239 19968 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.660072 19980 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.652340 20018 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.645330 20049 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.661857 20056 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:33.662804 20018 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.660130 19923 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.663017 20049 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.656442 19997 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.667820 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:33.668013 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.673624 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:33.674527 20035 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:04:33.675154 15159 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:04:33.675482 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:33.677232 15161 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:33.676779 19923 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.679895 19923 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.681233 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2229) took 1796 ms. Trace:
I20260709 13:04:33.681651 15230 rpcz_store.cc:276] 0709 13:04:31.885042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.500902 (+615860us) service_pool.cc:224] Handling call
0709 13:04:32.549295 (+ 48393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.556804 (+  7509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.556831 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.582251 (+ 25420us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.151694 (+569443us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:33.151732 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.151795 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.680961 (+529166us) raft_consensus.cc:1685] finished
0709 13:04:33.680981 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.681125 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3067,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":20829,"thread_start_us":476341,"threads_started":1,"wal-append.queue_time_us":826874}
I20260709 13:04:33.685591 19923 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.687328 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.675923 19930 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.689590 19930 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.691717 19930 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.660168 19968 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.681689 20027 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:33.693410 20027 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:33.694286 19975 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.694492 20027 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.696273 20027 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:33.697165 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.707603 15161 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:33.710104 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.725543 15154 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:33.712198 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:33.726100 15231 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:33.717298 15232 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:33.719261 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.720338 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.660740 19980 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.741123 20066 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.741377 19921 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:33.741736 20066 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.742904 19921 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:33.733974 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:33.707015 19598 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.735076 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:33.768049 15236 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:33.725061 20038 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:04:33.779542 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.780601 15159 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:33.739454 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:33.740145 19934 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.705736 20026 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.741204 19980 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.733181 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:33.781368 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.746129 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.748116 15160 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:33.765847 15230 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:33.778641 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:33.783708 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:33.784706 15237 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:33.796289 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:33.741199 19786 raft_consensus.cc:993] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: : Instructing follower 4b633b84351f40de9437e3d532fe6fa7 to start an election
I20260709 13:04:33.708734 19818 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:33.799238 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 182647us
I20260709 13:04:33.805611 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:33.810325 15232 raft_consensus.cc:2376] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11. Re-sending same reply.
I20260709 13:04:33.808959 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:33.823187 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 83820.9us
I20260709 13:04:33.823073 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.827639 15230 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:33.827577 15236 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:33.725450 20049 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:04:33.829151 15236 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:33.824898 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:33.840178 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.840713 15236 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:33.841056 15232 raft_consensus.cc:2376] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11. Re-sending same reply.
I20260709 13:04:33.834877 15237 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:33.839588 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:33.839325 20066 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.844113 19997 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:33.835839 20012 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:33.727978 20030 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.845098 20012 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:33.844135 20028 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:04:33.844748 19997 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.846027 20012 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.852097 20057 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.853736 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 128901us
I20260709 13:04:33.860380 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:33.860579 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:33.861712 15230 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:33.844575 20066 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:33.828608 20034 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.872668 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:33.876825 15082 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
W20260709 13:04:33.878671 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.863824 19996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.007s	sys 0.014s
I20260709 13:04:33.846787 19514 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.852146 20030 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.889456 20038 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:33.882543 20066 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:33.893033 20058 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:33.852731 20057 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.894884 19514 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:33.938730 19514 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:33.838594 19818 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:33.905241 19997 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:33.906190 19921 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.906666 20034 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:33.861747 19471 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.897552 20045 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:33.868687 20037 raft_consensus.cc:1081] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Signalling peer 4b633b84351f40de9437e3d532fe6fa7 to start an election
W20260709 13:04:33.881662 19996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.007s	sys 0.014s
I20260709 13:04:33.907012 19930 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.938895 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.896997 20031 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.946301 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:34.013700 20053 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.999598 19596 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.944881 20058 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:34.014304 20053 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:34.025010 20068 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:04:34.028954 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.030689 19813 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:34.005810 19930 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.045789 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.046106 19818 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.047112 20066 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.050467 20057 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:33.998302 19471 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.056133 20026 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.001765 20031 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.058238 19980 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.003449 19997 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:34.060807 20064 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:34.063699 20046 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.063799 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:33.950232 19452 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.068900 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.068593 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.075301 15088 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:04:34.077244 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.082993 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:34.104606 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.104278 19997 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.090834 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:34.109766 15843 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.110001 15154 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:04:34.112516 20031 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:34.094041 20008 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.053s	user 0.005s	sys 0.000s
W20260709 13:04:34.091852 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46195us
I20260709 13:04:34.064803 19595 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:34.080499 19813 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:34.079582 19514 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.079962 20027 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.066223 19596 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:34.091130 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 328461us
W20260709 13:04:34.114217 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 257046us
I20260709 13:04:34.101467 20054 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:33.953003 19921 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.180109 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.192138 15846 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:34.194025 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:34.195046 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.195325 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.196218 15231 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:34.154958 20015 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.137187 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1173) took 1671 ms. Trace:
I20260709 13:04:34.125268 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.180387 15848 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:34.138458 15843 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:04:34.213212 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 88087.9us
I20260709 13:04:34.131737 20054 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.144790 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.142378 19945 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.212868 15839 rpcz_store.cc:276] 0709 13:04:32.465282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.465391 (+   109us) service_pool.cc:224] Handling call
0709 13:04:32.481197 (+ 15806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.481542 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.481570 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.482029 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.712100 (+230071us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:32.735423 (+ 23323us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.735647 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.735731 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.698958 (+963227us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b180041a220. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:34.060457 (+361499us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:34.136931 (+ 76474us) raft_consensus.cc:1685] finished
0709 13:04:34.136955 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.137098 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27232,"prepare.queue_time_us":25635,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":2776,"raft.queue_time_us":53141,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":166,"spinlock_wait_cycles":39768192,"thread_start_us":229756,"threads_started":1,"wal-append.queue_time_us":314254}
I20260709 13:04:34.214619 15845 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:34.142174 20030 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.146318 20026 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:34.156392 19471 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:34.155092 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 72188us
I20260709 13:04:34.152035 20053 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.152565 19813 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.154244 19930 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.134835 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:34.232645 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 97889.8us
I20260709 13:04:34.155992 19596 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.089016 19915 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:04:34.156476 20027 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.203729 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:34.211608 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 415394us
I20260709 13:04:34.244050 15233 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:34.215848 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 999) took 1042 ms. Trace:
I20260709 13:04:34.216676 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.215466 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.218577 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.245235 15236 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:34.214301 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:34.219264 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.193468 20028 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.228336 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.225807 15847 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:34.203244 20031 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.233410 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.131760 19595 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:34.238751 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.244508 15234 rpcz_store.cc:276] 0709 13:04:33.173473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.173628 (+   155us) service_pool.cc:224] Handling call
0709 13:04:33.174070 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.581361 (+407291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.581412 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.582013 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.866789 (+284776us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:34.043258 (+176469us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b48002610d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:04:34.195558 (+152300us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.197974 (+  2416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.198092 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.198154 (+    62us) raft_consensus.cc:1685] finished
0709 13:04:34.215657 (+ 17503us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.215788 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":307878,"prepare.queue_time_us":177198,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":311096,"raft.run_cpu_time_us":3121,"raft.run_wall_time_us":568601,"spinlock_wait_cycles":11893504,"thread_start_us":649784,"threads_started":3,"wal-append.queue_time_us":444651}
I20260709 13:04:34.241361 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:34.212625 19871 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.214335 20054 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.242357 15087 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:34.192713 20058 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.225705 20026 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:34.131693 19514 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.225111 20075 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.221915 19980 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:34.221719 20022 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.245468 15841 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:34.249174 15085 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:34.250044 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:34.250254 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:34.226382 19818 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.234194 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.251916 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.253851 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:34.244789 15232 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:34.232519 20071 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.254997 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.238724 20053 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:34.315439 15847 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
W20260709 13:04:34.325585 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 75637.2us
W20260709 13:04:34.326011 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 100105us
I20260709 13:04:34.214725 19945 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.258883 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.252830 20031 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.326951 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.259402 15154 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:34.297907 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:34.314451 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 62621.6us
I20260709 13:04:34.312472 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.349541 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.253024 19813 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:34.323798 15843 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:34.327365 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.327579 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.252219 20012 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:34.287987 19659 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:34.359616 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:34.356907 15087 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:34.348570 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.350385 15083 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:34.353188 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.354341 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.360711 15840 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:34.362262 15230 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:34.362699 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.354317 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:34.356398 15080 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:34.230001 20030 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.255029 20059 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:34.362851 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 85997.1us
I20260709 13:04:34.363432 15839 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:34.353700 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58839.3us
I20260709 13:04:34.365540 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.328361 20022 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.296444 19773 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:04:34.352233 20031 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.350983 20041 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.363788 20053 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:34.379465 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52761us
I20260709 13:04:34.258587 19871 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.251590 20028 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.376119 19743 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.352411 19818 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.388404 15847 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:34.390740 15082 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:34.368124 19659 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.314030 20075 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.400980 20095 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:04:34.401263 20048 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.401715 19945 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.402279 19945 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:34.401902 20048 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.390483 19743 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.404466 20048 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.404780 19743 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.405997 20027 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.408001 19871 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.388517 19513 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:34.371105 20026 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.384135 20041 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.388228 20053 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.328819 20071 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.394500 20000 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.397382 19921 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.468114 15844 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:04:34.467203 20093 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:04:34.478176 19514 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.478461 20089 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.479314 19780 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:34.479092 20089 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.480559 19780 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:34.481822 19780 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.482622 20089 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.486220 20030 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.486757 19596 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:34.487349 19596 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:34.488451 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.491305 19999 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:34.491863 19999 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:34.492465 20041 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:34.493610 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.388875 20059 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:34.388464 19980 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:34.439994 19513 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:34.442400 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:34.499287 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.443733 19659 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.451012 20028 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.502971 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.419761 15839 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:34.503631 19513 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:34.494596 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.495703 20075 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.497310 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.511430 15844 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:34.498397 15080 pending_rounds.cc:85] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 5
I20260709 13:04:34.512454 15080 pending_rounds.cc:107] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:04:34.512804 15080 raft_consensus.cc:2889] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:34.511472 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.516129 15163 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:34.499578 15847 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:34.499972 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.501441 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.500612 20031 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.503702 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.517705 15085 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:34.509222 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:34.517719 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:34.519340 15079 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:34.494387 19999 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:34.499585 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136976us
I20260709 13:04:34.516829 15088 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:34.522396 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.524427 15160 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:34.524866 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.525398 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.525516 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.505103 19513 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.488323 19596 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.535019 15159 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:34.534901 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.536458 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.539978 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.525612 19735 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:34.523145 20012 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:34.540715 19907 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:34.547163 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 220309us
I20260709 13:04:34.547248 15844 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:34.548707 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.542752 19999 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.523033 20059 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:34.553535 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.541922 20053 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:34.558593 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 188178us
I20260709 13:04:34.558306 20095 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:34.566725 20095 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.567291 20095 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.567647 20012 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.572448 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.563400 19596 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:34.547348 20099 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:04:34.583827 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.561429 19907 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:04:34.572034 19818 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.561311 20059 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.552670 19980 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:34.584408 20017 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.570642 20104 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:04:34.647322 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 93895.6us
I20260709 13:04:34.656461 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.656775 19945 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.658699 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:34.661645 15234 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:34.661538 19945 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.663812 19945 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.598268 19788 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:34.659920 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.665138 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:34.657003 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.665779 15236 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:34.666178 15233 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:34.666992 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.667510 15233 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:34.668284 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.668498 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:34.669242 15232 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:34.669116 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.670156 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.658028 15231 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:34.671906 15231 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:34.673286 15231 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:34.670081 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:34.599495 20031 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.595185 20071 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:34.679296 15234 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:34.595057 20095 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:34.683415 15236 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
W20260709 13:04:34.685395 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.687736 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:34.690830 19942 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.691452 19942 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.691886 19942 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.697115 15231 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:34.698650 15231 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:34.701119 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:34.593964 20107 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.157s
I20260709 13:04:34.704421 15163 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:34.589274 20108 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.593876 20101 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.593660 20100 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.712011 15237 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:34.714490 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:34.707800 19513 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:34.717098 15228 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:34.679486 20017 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.720889 15845 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:04:34.721318 15845 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 9
I20260709 13:04:34.721546 15845 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:04:34.721801 15845 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:34.736233 15230 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:04:34.743796 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.708339 20054 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.763161 20054 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.746325 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:34.763657 19997 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.763865 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.764117 20101 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:34.764499 20101 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.764859 20101 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:34.766206 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:34.748060 19780 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.749867 19513 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.750473 19420 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:34.753297 20100 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:34.752925 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:34.758263 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.758348 15848 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:34.771040 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:34.774725 20054 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:34.775142 20054 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.775959 20054 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:34.770794 19420 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.779291 19420 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.771534 19513 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.779922 19942 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.781387 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.782066 19942 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:34.782397 15228 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d"
dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:43960
I20260709 13:04:34.783882 20022 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.785068 19513 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.785840 19927 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:34.773449 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.787074 19927 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:34.787461 19927 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.787774 19927 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.791319 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.770731 20095 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:34.799526 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:34.801227 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.801906 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:34.802518 15842 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:34.810789 19559 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:34.811583 19942 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:34.815006 20100 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:34.793782 19997 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.799497 20095 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.815795 20108 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:34.819984 20022 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:34.836387 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:34.834573 19997 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.839270 19997 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:34.840436 20108 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:34.844430 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.849331 20108 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:34.851516 15873 heartbeater.cc:646] Failed to heartbeat to 127.13.163.254:45133 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.13.163.254:45133 timed out after 15.000s (SENT)
I20260709 13:04:34.841701 20095 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:34.861265 19595 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:34.866688 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.887856 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:34.888599 15231 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.892715 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:34.893337 15155 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.897249 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.898396 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.887108 20017 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:34.905005 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.908039 15155 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:34.910487 15233 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:34.854321 19997 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.873662 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.874418 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.875097 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.914896 15160 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:34.876741 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.877462 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2289) took 1729 ms. Trace:
I20260709 13:04:34.879619 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:34.918421 15234 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:34.842594 20022 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.881165 15157 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:04:34.883631 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.868590 20107 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.909400 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:34.926079 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52484.1us
I20260709 13:04:34.927354 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.915746 15162 rpcz_store.cc:276] 0709 13:04:33.147754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.147882 (+   128us) service_pool.cc:224] Handling call
0709 13:04:33.156914 (+  9032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.180698 (+ 23784us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:33.743487 (+562789us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b18003c01c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9f75e5 00007f0d3f9f7568
0709 13:04:33.881872 (+138385us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.210772 (+328900us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.210890 (+   118us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:34.215753 (+  4863us) log.cc:844] Serialized 26 byte log entry
0709 13:04:34.215925 (+   172us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:34.216162 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.216358 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.225987 (+  9629us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.814199 (+588212us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b18003c01c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:34.871099 (+ 56900us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b18003c01c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:34.877270 (+  6171us) raft_consensus.cc:1685] finished
0709 13:04:34.877292 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.877410 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48803,"prepare.queue_time_us":324982,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":5275,"raft.queue_time_us":35512,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":16690,"spinlock_wait_cycles":21211648,"thread_start_us":441649,"threads_started":2,"wal-append.queue_time_us":360165}
I20260709 13:04:34.930640 15052 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.922736 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:34.935704 15231 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:34.936018 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:34.936591 15160 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.935704 15163 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:34.940147 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:34.903328 20054 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.945775 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.948516 15156 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:34.947896 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:34.947877 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:34.942329 19780 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.924440 20107 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.954234 20101 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.932274 20022 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:34.942098 20015 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:34.942507 20017 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:34.942533 19958 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:34.976070 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:34.952654 20107 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.979971 15160 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:34.981123 15158 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:34.983269 20015 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:34.946308 19513 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:34.983973 19513 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.984338 15156 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:34.984208 20015 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.948800 19995 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.946564 19997 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:34.979329 20017 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:34.986481 19780 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:34.984380 19513 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:34.957192 19813 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:04:34.996191 20033 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:35.020006 19927 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.008241 20017 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:35.034855 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.030761 20101 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.014770 20033 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.029224 19780 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:35.033304 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:35.051247 15848 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.007718 19958 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:04:35.045316 20033 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:35.037818 20037 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:04:35.061375 19497 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:35.079376 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51357.5us
I20260709 13:04:35.071307 20066 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:35.083829 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.087231 15842 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:35.091540 15848 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:35.102236 15228 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:35.102722 15228 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:35.083405 20066 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:35.107268 20066 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.110098 20066 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:35.112447 20122 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:35.083566 20019 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:04:35.083945 20054 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.125129 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.124971 19780 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.125521 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.129170 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.129899 15845 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:35.130007 15230 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:35.131302 19927 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.133251 19997 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:35.141264 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.155947 20123 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:04:35.141616 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.160676 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.162698 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.163326 15234 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:35.164175 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.165357 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.165992 15230 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:04:35.167536 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.168965 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.174778 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.175588 15234 raft_consensus.cc:2368] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:35.186174 19813 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:35.168745 20108 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.165820 19664 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.215056 19664 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.165320 20046 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:35.216442 20046 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.217613 19664 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.218814 20046 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.219696 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.213714 20019 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.165843 20065 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:35.214931 20108 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.221828 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.181262 15228 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:35.233268 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.235181 20071 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:35.235798 20071 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:35.236644 20071 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.237602 19513 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.223536 20022 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.223644 20045 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:04:35.241783 20022 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.234051 20015 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:35.242777 20015 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.243181 20015 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.225085 20065 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.234596 20107 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.220548 20019 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.248558 20019 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.251250 20107 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.252777 19596 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:35.240227 19927 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:04:35.253140 15162 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:35.253506 19596 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.254388 20102 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.214434 19813 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.254949 20102 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.255600 19596 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:35.256548 19813 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:35.256939 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.257692 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1244) took 1038 ms. Trace:
I20260709 13:04:35.258105 15841 rpcz_store.cc:276] 0709 13:04:34.219188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.361479 (+142291us) service_pool.cc:224] Handling call
0709 13:04:34.361779 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:34.453149 (+ 91370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.453182 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.453257 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:34.453321 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.257439 (+804118us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b54006d4054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:35.257496 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.257633 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209695104}
I20260709 13:04:35.261695 19997 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:35.262236 19997 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.226395 19664 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:35.240166 19513 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:35.292418 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.328715 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:35.266453 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 124929us
I20260709 13:04:35.234602 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.265265 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.337484 15155 raft_consensus.cc:2376] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12. Re-sending same reply.
I20260709 13:04:35.269912 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:35.266305 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:35.264484 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.338516 15841 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:35.338884 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.336531 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.342649 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.342825 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.343266 15157 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:35.265947 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.345408 15161 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:35.346371 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.346947 15161 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:35.347749 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:35.264885 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.281741 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:35.349316 15158 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:35.350847 15161 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:35.352021 15161 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:35.357447 15154 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:35.358224 19871 raft_consensus.cc:1064] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: attempting to promote NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 to VOTER
I20260709 13:04:35.358282 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.360311 15161 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:35.360680 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 126167us
I20260709 13:04:35.267120 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.265605 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.348028 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.348276 15156 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:35.266538 20011 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:35.363915 19958 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:35.366628 20095 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.266237 20028 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.361817 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.264200 19997 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.263931 20108 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.372349 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:35.335629 20102 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.335887 19664 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:35.385667 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.385200 19780 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.386389 20038 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:35.359776 20065 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:35.362844 20012 raft_consensus.cc:2764] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:35.389247 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.362214 19999 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:35.291051 20046 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:04:35.369052 20017 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.371713 20011 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:35.407732 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.377159 20022 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:35.407174 19780 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.416997 20022 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.417704 20022 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.419127 19780 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.409875 15231 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:35.386276 19513 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:04:35.405243 19664 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.392633 19958 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:35.404971 20038 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:35.426538 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54282.2us
I20260709 13:04:35.427057 15051 leader_election.cc:400] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:35.427402 15051 leader_election.cc:403] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:35.427294 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.430274 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 995) took 2256 ms. Trace:
I20260709 13:04:35.422380 19559 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:35.433425 19999 raft_consensus.cc:2764] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:04:35.433516 19559 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.430666 15235 rpcz_store.cc:276] 0709 13:04:33.168333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.181202 (+ 12869us) service_pool.cc:224] Handling call
0709 13:04:33.223774 (+ 42572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.487551 (+263777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.487621 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.488305 (+   684us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.604507 (+116202us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:35.118928 (+1514421us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b18003d8400. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf
0709 13:04:35.119068 (+   140us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.420509 (+301441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.421090 (+   581us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:35.421134 (+    44us) raft_consensus.cc:1685] finished
0709 13:04:35.424265 (+  3131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.424426 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272401,"prepare.queue_time_us":329724,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3582,"raft.queue_time_us":263238,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":20321,"spinlock_wait_cycles":8522624,"thread_start_us":745521,"threads_started":3,"wal-append.queue_time_us":284765,"wal-append.run_cpu_time_us":2794,"wal-append.run_wall_time_us":1144285}
I20260709 13:04:35.421006 20011 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.393330 20028 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.428155 20075 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:35.429345 20038 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.442907 20022 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:35.460317 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 107083us
I20260709 13:04:35.462440 15841 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:35.461625 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:35.466847 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.449891 20075 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.480228 20017 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.482091 20022 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.482203 20033 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.466002 20046 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.466423 20012 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:35.474123 20075 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:35.503682 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.504634 15082 raft_consensus.cc:2410] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:35.505470 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.510752 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.511377 15082 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.523327 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:35.524184 15156 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.528232 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:35.531963 20129 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:04:35.533252 15083 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:04:35.535101 20131 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:04:35.535321 20033 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.538092 15082 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:35.552119 19559 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:35.553061 20128 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:04:35.554168 20132 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:35.483614 20003 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:04:35.561877 20028 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:35.563951 20130 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:04:35.567245 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.574364 15088 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:35.575131 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.575385 20015 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.577579 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.522688 20135 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:35.579022 20135 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.579216 19596 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:35.579422 20135 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.575397 15080 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:35.575743 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.591543 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.586095 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.599257 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.592496 15235 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:35.604197 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.604827 15237 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:35.607132 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.608563 15841 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:35.609257 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:35.610630 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.607772 15157 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:35.613878 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:35.614243 15235 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:35.617668 15157 raft_consensus.cc:2368] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:35.621817 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.623034 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.623581 15229 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:35.624297 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.625126 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.625803 15237 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:35.629565 15234 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:35.632246 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.634066 15228 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:35.637589 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:35.638417 15233 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 7.
I20260709 13:04:35.579676 19596 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:35.642033 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.596092 20095 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.650259 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.646931 15228 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:04:35.617101 20108 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:35.647665 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.660668 15087 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:35.660760 15236 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:35.651417 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:35.661780 15235 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:35.642580 15054 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.13.163.254:45133, user_credentials={real_user=slave}} blocked reactor thread for 73100.5us
I20260709 13:04:35.642241 15231 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:35.669466 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.671339 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.671756 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.672668 15156 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:35.673003 15087 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:35.673338 15087 pending_rounds.cc:85] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 9
I20260709 13:04:35.673532 15087 pending_rounds.cc:107] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:35.673584 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:35.673820 15087 raft_consensus.cc:2889] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:35.674173 15156 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:35.677196 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45252.6us
I20260709 13:04:35.680137 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.678069 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.680311 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:35.682708 15848 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:35.682766 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.683387 15157 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:35.684809 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.686475 15081 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:35.689239 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:35.689360 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 221280us
I20260709 13:04:35.696931 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:35.697796 15157 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:35.661327 19971 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.700666 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50473.5us
I20260709 13:04:35.691015 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.635933 20101 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:04:35.686673 20033 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:04:35.642767 20037 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:04:35.712752 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:35.631582 19420 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:35.715421 15079 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:35.725579 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.716641 15157 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:04:35.705025 20108 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:35.728691 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:35.696563 20136 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:35.731570 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:35.714023 20017 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:35.743666 15156 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:35.717579 20003 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:35.739580 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:35.830140 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 90660us
I20260709 13:04:35.831214 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:35.686484 19930 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:04:35.725809 20053 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:35.832195 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.736474 15160 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:35.834583 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 121906us
I20260709 13:04:35.842998 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.843577 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.844112 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.752553 15235 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:35.741218 20017 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.732053 19769 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:35.849462 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.865844 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.877485 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.878407 15084 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.719655 19971 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.738678 20137 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:35.910715 20137 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:35.833930 19924 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:35.911177 20137 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:35.911337 19924 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.911765 19924 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:35.843641 20028 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:35.886754 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.887128 15839 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 13. Ops: []
I20260709 13:04:35.887601 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.866460 20014 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:35.913445 20014 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:35.890883 19971 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:35.913825 20014 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.842069 20012 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:35.839777 19818 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:35.914589 19818 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:35.897951 20140 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:35.898115 19930 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:35.728693 19596 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.915444 19818 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.917790 19596 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:35.917918 19818 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:35.865707 20017 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:35.923295 19871 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 5, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.925981 20038 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.903264 20046 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10.
I20260709 13:04:35.904595 20135 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.868163 20053 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:35.880702 19999 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:35.906615 19664 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.906960 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.833707 20136 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:35.907671 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.941948 15081 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:35.883443 20066 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:04:35.906656 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:35.943212 15084 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:35.908418 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:35.909159 20133 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:04:35.909993 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:35.917485 20075 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:35.898869 20108 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:35.937537 15082 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:35.947286 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:35.969386 15054 leader_election.cc:400] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:35.969771 15054 leader_election.cc:403] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:35.959369 20136 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:35.970834 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.972659 15845 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:35.986615 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.000965 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:36.001657 15839 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:36.009382 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:35.952462 20015 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:35.953897 20033 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:36.010246 15842 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:36.046303 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.051390 15839 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:36.057943 15085 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:35.945019 20011 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.062265 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:36.063887 20011 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:36.065847 15154 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:36.068359 15848 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:36.073628 20038 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.074620 20038 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.075423 20038 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:36.083333 20135 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.090461 20053 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.056995 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:35.954509 20019 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
W20260709 13:04:36.106767 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49869.1us
I20260709 13:04:36.112460 15079 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:36.044133 19999 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:36.118235 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 280262us
I20260709 13:04:36.121215 15845 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:36.122793 15848 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:36.124868 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:36.125545 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:36.130535 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 494282us
W20260709 13:04:36.128114 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 450112us
I20260709 13:04:36.143865 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:36.116350 20033 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:36.162779 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:36.164887 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 321512us
W20260709 13:04:36.137609 19997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.027s	sys 0.064s
I20260709 13:04:36.171216 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:36.180927 15203 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.517s (SENT)
I20260709 13:04:36.144462 20026 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:36.125573 20141 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:36.174208 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:36.175725 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:36.180404 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:36.185109 15229 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.012s	sys 0.004s
I20260709 13:04:36.165359 19497 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:36.206530 15229 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.013s	sys 0.005s
W20260709 13:04:36.206900 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.193004 15203 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.529s (SENT)
I20260709 13:04:36.207001 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:36.164232 20058 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.211066 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:36.128756 20011 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:04:36.168205 20140 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:36.159749 20107 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.165194 19664 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.125039 19999 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:36.128050 20135 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:04:36.207528 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:36.188108 15160 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:36.178573 20012 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:36.230397 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:36.211629 20003 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:36.245790 15845 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:04:36.177521 19997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.027s	sys 0.064s
I20260709 13:04:36.244755 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:36.248394 15155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.016s	sys 0.011s
I20260709 13:04:36.211800 20028 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:36.248801 15155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.016s	sys 0.011s
I20260709 13:04:36.214387 20033 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:36.251549 15844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.009s	sys 0.019s
I20260709 13:04:36.178943 19420 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:36.257416 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:36.258033 15844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.009s	sys 0.020s
I20260709 13:04:36.258380 15084 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:36.258878 15080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.010s	sys 0.019s
W20260709 13:04:36.259268 15080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.011s	sys 0.019s
I20260709 13:04:36.216886 20141 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:36.227460 19497 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:36.262181 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:36.208956 19958 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:04:36.219549 20102 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.007s
I20260709 13:04:36.245623 20003 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.277191 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:36.249248 20140 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:36.278894 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:36.248754 19769 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:36.280972 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:36.284962 15081 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:36.256549 20028 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:36.289738 15237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.026s	sys 0.039s
I20260709 13:04:36.226503 20058 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:36.293736 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 324515us
I20260709 13:04:36.246235 20107 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.273012 19971 raft_consensus.cc:1027] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: attempt to promote peer 3c28be6f86ea4fb6b79a47d3999f5a82: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:04:36.265741 20157 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:36.249019 20059 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:36.327493 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:36.276325 20046 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.283632 19420 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:36.297567 20102 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.012s	sys 0.008s
I20260709 13:04:36.265137 20033 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:36.294660 20075 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.275336 20054 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:04:36.364931 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.462276 20028 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:36.417364 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 140249us
W20260709 13:04:36.622007 15127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.533s (SENT) [suppressed 28 similar messages]
I20260709 13:04:36.341833 20038 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:36.342456 19599 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:04:36.425273 19997 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:36.435669 20155 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:36.418073 20101 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:36.419479 20141 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:36.449251 20149 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:04:36.436425 20145 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 13:04:36.445928 19999 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.446070 19769 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:36.443900 20014 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.000s
W20260709 13:04:36.440562 19596 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.000s
I20260709 13:04:36.324250 20159 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.457643 20140 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:36.426385 15237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.026s	sys 0.039s
I20260709 13:04:36.463896 15084 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:36.327198 20071 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.339311 19942 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:36.440778 20144 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:04:36.420198 20003 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:36.582800 20022 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.565120 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.728287 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 266378us
W20260709 13:04:36.565318 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 372351us
I20260709 13:04:36.730139 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:36.730587 15203 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.567s (SENT)
I20260709 13:04:36.731351 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:36.297698 15083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.010s	sys 0.008s
W20260709 13:04:36.732321 15083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.011s	sys 0.008s
I20260709 13:04:36.462584 20153 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.468287 20157 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.469959 20158 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:36.652613 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1033) took 1011 ms. Trace:
W20260709 13:04:36.664094 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52848.2us
I20260709 13:04:36.625572 20012 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.630280 19813 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:36.685789 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 478303us
W20260709 13:04:36.689229 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.035s	sys 0.076s
W20260709 13:04:36.750618 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.035s	sys 0.076s
I20260709 13:04:36.563437 20046 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:36.461474 19497 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.653798 20019 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.695470 15084 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:36.696817 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 415878us
I20260709 13:04:36.766283 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:36.767139 15231 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:36.653546 20038 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.723953 20101 raft_consensus.cc:476] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:36.738854 15054 leader_election.cc:400] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
W20260709 13:04:36.726751 20017 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.001s
W20260709 13:04:36.746392 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124423us
W20260709 13:04:36.777233 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.743630 15229 rpcz_store.cc:276] 0709 13:04:35.640747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.642459 (+  1712us) service_pool.cc:224] Handling call
0709 13:04:35.642722 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.642900 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:36.210298 (+567398us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.652250 (+441952us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:36.652305 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:36.652354 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.652422 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.652555 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":634704,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":161161,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":45282,"thread_start_us":987230,"threads_started":2}
W20260709 13:04:36.725518 20045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.006s	sys 0.020s
I20260709 13:04:36.776142 20019 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.788328 20017 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:36.757485 20012 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.726009 19942 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.747309 20157 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:36.751058 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 562429us
I20260709 13:04:36.822880 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1049) took 1482 ms. Trace:
I20260709 13:04:36.823400 15154 rpcz_store.cc:276] 0709 13:04:35.340896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.358243 (+ 17347us) service_pool.cc:224] Handling call
0709 13:04:35.358503 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.066352 (+707849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.793817 (+727465us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54001266d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:36.822789 (+ 28972us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626205440}
I20260709 13:04:36.745878 20059 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:36.828239 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:36.754467 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:36.756234 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:36.763316 20053 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.761639 15080 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:36.724776 20173 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:36.768683 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:36.771229 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2109) took 1256 ms. Trace:
I20260709 13:04:36.773018 15054 leader_election.cc:403] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:36.693665 20014 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:36.766702 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:36.766535 15084 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:36.788357 20045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.006s	sys 0.020s
I20260709 13:04:36.759290 20158 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.714082 20159 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.800729 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2284) took 1639 ms. Trace:
I20260709 13:04:36.741914 20022 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.745903 20153 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.748502 15235 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:37.057183 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:36.737062 20071 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.814160 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2794) took 1554 ms. Trace:
I20260709 13:04:36.814615 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1052) took 1472 ms. Trace:
I20260709 13:04:36.815228 20046 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.719048 19420 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:36.791741 20075 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:36.818432 20107 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:36.819077 20033 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.820747 20017 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:36.723567 20166 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.829979 15233 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:36.831444 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77101.6us
I20260709 13:04:36.838363 15231 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:04:36.840379 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.197s	user 0.021s	sys 0.022s
I20260709 13:04:37.132328 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:36.820578 20137 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.909s	user 0.008s	sys 0.002s
W20260709 13:04:36.845610 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78980.5us
W20260709 13:04:36.847172 15232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.001s	sys 0.041s
I20260709 13:04:37.056681 20065 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.031620 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 203449us
I20260709 13:04:36.841677 15086 rpcz_store.cc:276] 0709 13:04:35.514915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.515061 (+   146us) service_pool.cc:224] Handling call
0709 13:04:35.515399 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.527819 (+ 12420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.527851 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.533617 (+  5766us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.967417 (+433800us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:35.967472 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.967559 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.770933 (+803374us) raft_consensus.cc:1685] finished
0709 13:04:36.770963 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.771159 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12056,"raft.queue_time_us":18811,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":65,"thread_start_us":433467,"threads_started":1,"wal-append.queue_time_us":592981}
W20260709 13:04:37.033993 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 197423us
W20260709 13:04:36.834638 20160 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 13:04:36.834434 20156 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:04:37.034741 15230 rpcz_store.cc:276] 0709 13:04:35.160855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.166582 (+  5727us) service_pool.cc:224] Handling call
0709 13:04:35.211069 (+ 44487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.214123 (+  3054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.214171 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.562744 (+348573us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b2c00032e28. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:04:35.612289 (+ 49545us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.128354 (+516065us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:36.128409 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.128505 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.765467 (+636962us) raft_consensus.cc:1685] finished
0709 13:04:36.765502 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.800641 (+ 35139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2785,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":134,"spinlock_wait_cycles":16410368,"thread_start_us":460404,"threads_started":1,"wal-append.queue_time_us":622761}
I20260709 13:04:36.826217 20066 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.057926 15157 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:36.829216 19813 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:37.179209 19813 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.182245 19813 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.185562 20167 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.186349 20167 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:36.839728 19924 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.928s	user 0.001s	sys 0.011s
I20260709 13:04:37.192425 19924 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.193143 20066 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.195366 19924 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.197078 20153 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.197580 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2108) took 1687 ms. Trace:
I20260709 13:04:37.197963 15085 rpcz_store.cc:276] 0709 13:04:35.510140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.510444 (+   304us) service_pool.cc:224] Handling call
0709 13:04:35.510702 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.060975 (+550273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.197350 (+1136375us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:37.197522 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257925376}
I20260709 13:04:37.198506 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2295) took 1578 ms. Trace:
I20260709 13:04:37.198886 15237 rpcz_store.cc:276] 0709 13:04:35.620440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.626436 (+  5996us) service_pool.cc:224] Handling call
0709 13:04:35.626716 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.678736 (+ 52020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.678780 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.687528 (+  8748us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.695086 (+1007558us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:36.695148 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.695243 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.198316 (+503073us) raft_consensus.cc:1685] finished
0709 13:04:37.198335 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.198453 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29388,"raft.queue_time_us":50077,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":15435,"thread_start_us":967292,"threads_started":1,"wal-append.queue_time_us":852818}
I20260709 13:04:37.058574 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.059346 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.202512 15158 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:37.062711 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.203945 15154 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.060382 15161 rpcz_store.cc:276] 0709 13:04:35.341762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.361338 (+ 19576us) service_pool.cc:224] Handling call
0709 13:04:35.361595 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.422721 (+ 61126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.422755 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.452504 (+ 29749us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.964763 (+512259us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:35.964820 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.964934 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.814378 (+849444us) raft_consensus.cc:1685] finished
0709 13:04:36.814406 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.814555 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21123,"raft.queue_time_us":44885,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":441855,"threads_started":1,"wal-append.queue_time_us":645049}
I20260709 13:04:37.060251 15155 rpcz_store.cc:276] 0709 13:04:35.259639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.338088 (+ 78449us) service_pool.cc:224] Handling call
0709 13:04:35.359483 (+ 21395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.493470 (+133987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.493502 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.494073 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.254330 (+760257us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:36.254399 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.254490 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.440318 (+185828us) raft_consensus.cc:1685] finished
0709 13:04:36.440362 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.798641 (+358279us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b54003e0b50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:36.814073 (+ 15432us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20653,"raft.queue_time_us":30211,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4939,"spinlock_wait_cycles":12836608,"thread_start_us":793668,"threads_started":2,"wal-append.queue_time_us":927517}
I20260709 13:04:37.071895 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2112) took 1438 ms. Trace:
W20260709 13:04:37.205263 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 476036us
I20260709 13:04:36.871233 20053 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.207011 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:37.074189 15088 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:36.844727 20101 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:37.112516 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.034723 20136 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:37.069157 20170 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 13:04:37.133606 15232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.001s	sys 0.041s
I20260709 13:04:37.218899 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2293) took 1599 ms. Trace:
W20260709 13:04:36.848546 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 109891us
I20260709 13:04:37.222506 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:37.132732 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.490s	user 0.022s	sys 0.022s
I20260709 13:04:37.134642 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:37.226480 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:37.060230 19997 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.110082 20152 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.086666 20171 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.103338 20075 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.303665 15847 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:37.103914 20059 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.163865 19599 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:04:37.175202 20022 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.176697 20172 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.178058 19971 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.205591 15083 rpcz_store.cc:276] 0709 13:04:35.633241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.651580 (+ 18339us) service_pool.cc:224] Handling call
0709 13:04:35.682124 (+ 30544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.682337 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:35.705409 (+ 23072us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.071321 (+1365912us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.071506 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.071554 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:37.071611 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.071689 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.071824 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23318,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3106,"raft.queue_time_us":93672,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":53468,"thread_start_us":1361692,"threads_started":2,"wal-append.queue_time_us":959886}
W20260709 13:04:37.212105 15082 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.042s	sys 0.041s
W20260709 13:04:37.326272 15082 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.042s	sys 0.041s
I20260709 13:04:37.137111 20058 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:37.233868 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 99306.2us
I20260709 13:04:37.327512 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:37.329188 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:37.178114 20174 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 13:04:37.219445 15232 rpcz_store.cc:276] 0709 13:04:35.619904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.625858 (+  5954us) service_pool.cc:224] Handling call
0709 13:04:35.645520 (+ 19662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.645758 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:36.144044 (+498286us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.218417 (+1074373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.218491 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:37.218561 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.218653 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.218823 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14402,"prepare.queue_time_us":672602,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":1531405,"threads_started":2}
I20260709 13:04:37.031616 20173 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.285759 15844 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:37.285295 19664 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.084901 20178 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 13:04:37.291680 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:37.143745 20187 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:37.167520 20017 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:37.299998 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:37.301921 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:37.114670 20046 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:37.150033 20108 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.136s
W20260709 13:04:37.307371 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.207886 19818 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.317173 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:37.335770 19871 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:37.153439 20065 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.166323 20107 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:04:37.312680 15154 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:04:37.325834 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 103432us
I20260709 13:04:37.373974 15229 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:37.340286 20012 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:37.134254 20095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.005s	sys 0.012s
I20260709 13:04:37.336018 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:37.343609 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 202675us
I20260709 13:04:37.458740 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:37.152938 20137 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:37.313726 20075 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:37.305526 20038 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.323837 20153 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:37.336896 20160 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:37.152670 20166 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.314095 20045 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:37.350364 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 210256us
I20260709 13:04:37.332026 20022 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.211261 20101 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.353896 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1063) took 1301 ms. Trace:
W20260709 13:04:37.338718 20141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.020s	sys 0.031s
W20260709 13:04:37.375901 15081 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.031s
I20260709 13:04:37.374513 20017 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.339231 20183 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.339181 20058 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:37.286303 19975 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:37.379395 19818 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.305078 19997 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.468039 15086 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:37.351343 20046 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.305464 20152 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.376188 19769 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:37.453760 20026 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.249s	user 0.013s	sys 0.000s
I20260709 13:04:37.483152 20167 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.483327 20173 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.382846 19871 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.484306 20159 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.485817 20053 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.486323 20137 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:37.486656 20182 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:04:37.389693 19942 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.580566 19942 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:37.581126 19942 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.583205 19942 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.583674 19942 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.584086 19942 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.486927 15162 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:37.323066 20171 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:37.466893 20095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.005s	sys 0.013s
I20260709 13:04:37.495091 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.501237 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:37.591197 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90052.6us
I20260709 13:04:37.349553 19664 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.594460 19924 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:37.594758 19664 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.596225 20173 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.596763 20173 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.598902 20173 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.599411 20173 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.599875 20173 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:37.508191 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.825901 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.602540 20066 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.603042 20066 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.603454 20066 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.603825 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:37.515074 15081 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.010s	sys 0.031s
I20260709 13:04:37.307109 20136 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:04:37.607618 19975 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.608132 19975 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:37.608790 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.609627 15079 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:04:37.610587 19780 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:37.611002 19780 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:37.611843 19780 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.457245 20033 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.511425 15159 rpcz_store.cc:276] 0709 13:04:36.052484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.053499 (+  1015us) service_pool.cc:224] Handling call
0709 13:04:36.053921 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.054331 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.054372 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.054911 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.374721 (+319810us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:36.374785 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.374881 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.353713 (+978832us) raft_consensus.cc:1685] finished
0709 13:04:37.353731 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.353843 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3294,"thread_start_us":255814,"threads_started":1,"wal-append.queue_time_us":559109}
I20260709 13:04:37.472360 20025 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:37.525795 20011 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:04:37.471946 20188 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.328214 20059 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.491595 20014 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.688465 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.502810 19559 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.493860 20019 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.497169 19599 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:37.520630 20141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.020s	sys 0.032s
I20260709 13:04:37.622097 20137 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.513422 20153 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:37.535876 20183 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.513630 19930 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:37.457669 20187 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:04:37.477079 20185 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:04:37.574307 15235 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:37.545776 20046 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.537899 20158 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.515993 20045 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.573549 19871 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:37.783250 20166 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.514989 20022 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.464777 20165 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.784499 20165 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.785053 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.785737 20022 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.786857 20165 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.787177 20022 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.787750 20022 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.551448 20108 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.615029 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.790158 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.615792 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.791266 15154 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:37.612789 15079 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:37.618256 15844 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:37.619010 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.620186 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.794559 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.620798 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:37.624496 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 171323us
I20260709 13:04:37.797641 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:37.799088 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:37.624322 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.633066 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.588619 20171 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.559501 20139 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.558451 20071 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:37.681999 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 346049us
I20260709 13:04:37.684975 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2301) took 1515 ms. Trace:
I20260709 13:04:37.810751 15228 rpcz_store.cc:276] 0709 13:04:36.169445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.169640 (+   195us) service_pool.cc:224] Handling call
0709 13:04:36.169970 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.170335 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.170365 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.170845 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.468275 (+297430us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:36.468314 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.468393 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.684673 (+1216280us) raft_consensus.cc:1685] finished
0709 13:04:37.684702 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.684901 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":232363,"threads_started":1,"wal-append.queue_time_us":527814}
I20260709 13:04:37.689090 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.690171 15155 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:04:37.692058 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2116) took 1477 ms. Trace:
I20260709 13:04:37.773953 20187 consensus_queue.cc:1243] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:37.335819 20172 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:37.613477 20180 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 13:04:37.700830 15847 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:37.562650 20058 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:04:37.726740 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.586387 19942 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.758584 20137 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:37.522224 20003 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.778s	user 0.006s	sys 0.005s
I20260709 13:04:37.682842 20026 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.765972 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.712076 19780 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:37.613098 20033 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.763584 20014 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.728133 19599 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.795722 15842 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:37.911847 15163 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:37.807287 15159 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:37.808444 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:37.846011 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.847410 15080 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:37.818602 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.802538 20108 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.848392 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:37.926831 15087 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:37.849431 15840 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:37.929576 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81352.1us
I20260709 13:04:37.857671 20171 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:04:37.850055 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.695822 15084 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:37.854415 15841 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:37.851809 15088 rpcz_store.cc:276] 0709 13:04:36.215109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.215643 (+   534us) service_pool.cc:224] Handling call
0709 13:04:36.215973 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.208136 (+992163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.691775 (+483639us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54003cf354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:37.691990 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973134592}
W20260709 13:04:37.857156 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.007s	sys 0.011s
I20260709 13:04:37.680430 20025 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.734030 19559 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.874426 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.727626 20188 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.519392 20101 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:37.929271 20071 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:37.885552 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.890252 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.933163 19997 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.776059 20153 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:04:37.910796 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:37.856112 20107 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.982223 20153 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:37.982637 20153 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:37.982441 20107 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.983497 20153 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.984510 20107 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.881135 20199 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.985285 20199 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.987414 20199 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.923244 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:37.988588 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 65466.2us
I20260709 13:04:37.927594 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.992682 15842 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:37.886111 20140 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:37.909863 20066 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.922700 20014 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:37.932018 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 307196us
W20260709 13:04:37.936959 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.007s	sys 0.011s
W20260709 13:04:38.004787 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 798637us
W20260709 13:04:37.949011 20028 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.295s	user 0.011s	sys 0.002s
I20260709 13:04:38.008100 20165 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:37.917083 19599 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:38.008414 20028 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.008697 20165 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.941511 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.942564 15084 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:37.887881 19942 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:04:37.943563 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:37.889547 20026 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:37.880275 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.946460 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.946815 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:37.887163 20003 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.930184 20017 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.905696 15162 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:37.954203 19559 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.955127 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:38.037159 20179 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.376s	user 0.000s	sys 0.001s
I20260709 13:04:38.037341 15847 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
W20260709 13:04:38.037748 20195 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:04:38.030661 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.038369 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.955986 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.039376 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:38.039844 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:37.958125 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.040434 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.040877 15079 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:04:38.040922 20206 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:04:38.041268 20217 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:04:38.041808 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:38.042321 20207 consensus_queue.cc:1175] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 9 status { last_received { term: 9 index: 11 } last_committed_idx: 10 last_received_current_leader { term: 9 index: 11 } }
I20260709 13:04:38.043139 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:37.959859 20025 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.968114 20137 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.970355 20033 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.979099 20022 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.937880 20173 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.979941 20059 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:37.930080 20045 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:37.981395 20108 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.920585 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:37.984710 20153 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:37.987694 20172 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:37.937736 20058 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:37.993813 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2744) took 1714 ms. Trace:
I20260709 13:04:37.849205 20139 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:37.994844 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.056124 15233 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:04:38.008116 15839 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:04:37.996628 20152 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.034168 15840 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:38.034926 15230 raft_consensus.cc:2368] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14.
I20260709 13:04:38.036276 20204 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:38.036716 20210 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.009694 19497 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:04:38.065636 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2114) took 1852 ms. Trace:
I20260709 13:04:37.996379 19871 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.066044 15082 rpcz_store.cc:276] 0709 13:04:36.212913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.213090 (+   177us) service_pool.cc:224] Handling call
0709 13:04:36.213424 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.650660 (+437236us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:37.326629 (+675969us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.762839 (+436210us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b1800259540. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:04:37.763057 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.764065 (+  1008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.764114 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:37.764166 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.788103 (+ 23937us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b1800259540. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:38.065428 (+277325us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:38.065471 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.065582 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41659,"prepare.queue_time_us":925646,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":2608,"raft.queue_time_us":10196,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":10003,"spinlock_wait_cycles":1023296256,"thread_start_us":877964,"threads_started":2,"wal-append.queue_time_us":449254}
I20260709 13:04:38.067691 15155 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:38.008822 20183 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.041491 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.040795 15237 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:38.050985 15231 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:04:37.949468 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 223904us
W20260709 13:04:38.047340 20203 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 13:04:38.004297 19930 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.010913 15228 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:38.009697 15160 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:38.058817 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64048.3us
I20260709 13:04:38.081823 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:38.082854 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.083189 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.084197 15231 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:38.089011 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.089798 15230 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:38.085767 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.091236 15231 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:38.005641 20199 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:04:38.056706 20200 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.042135 20205 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:04:38.060796 20137 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:38.061874 20204 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.030932 20011 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 13:04:38.044854 20197 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 13:04:38.068277 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.039911 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:37.978286 20157 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.157s	user 0.011s	sys 0.000s
I20260709 13:04:38.072120 15079 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:38.074893 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.053491 15081 rpcz_store.cc:276] 0709 13:04:36.279851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.435361 (+155510us) service_pool.cc:224] Handling call
0709 13:04:36.435721 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.828686 (+392965us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:37.605495 (+776809us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.857899 (+252404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.857977 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:37.858327 (+   350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.993606 (+135279us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54000988d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:37.993650 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.993762 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245559,"prepare.queue_time_us":680925,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":17852,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8376,"spinlock_wait_cycles":869735808,"thread_start_us":1005960,"threads_started":2,"wal-append.queue_time_us":332091}
I20260709 13:04:38.077778 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:38.075910 15088 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:38.053056 20058 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:38.094587 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:38.101326 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 300867us
I20260709 13:04:38.058979 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.106559 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.107028 15230 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:38.107982 15233 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:38.110996 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.111254 15158 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:38.006603 20140 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.115649 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.117419 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.121179 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:38.121999 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.328490 15154 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
W20260709 13:04:38.329571 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 208505us
W20260709 13:04:38.330551 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 236018us
I20260709 13:04:38.331385 15162 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 9: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:38.123590 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.181391 15231 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:38.338531 15083 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:04:38.339000 15083 pending_rounds.cc:85] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 10
I20260709 13:04:38.339215 15083 pending_rounds.cc:107] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:04:38.339525 15083 raft_consensus.cc:2889] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.080973 20101 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.080653 19871 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.088009 20014 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.336665 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:38.362100 20046 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.549s	user 0.000s	sys 0.018s
I20260709 13:04:38.363579 20204 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:38.374831 20075 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.884s	user 0.009s	sys 0.004s
I20260709 13:04:38.341526 20183 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:38.364696 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:38.123063 20200 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.369125 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2306) took 1619 ms. Trace:
I20260709 13:04:38.370538 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.371598 20210 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:38.191943 19599 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.137245 20028 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.336259 20165 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:38.374567 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 296864us
I20260709 13:04:38.415841 20017 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:38.382892 20173 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:38.376449 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.377292 15082 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:38.377643 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:38.237233 19975 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.008s
W20260709 13:04:38.330083 20205 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 13:04:38.381091 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 363407us
I20260709 13:04:38.112553 19924 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:38.379768 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 283482us
I20260709 13:04:38.389776 15080 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:38.390328 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.335512 20058 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:38.336786 20157 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.082247 20015 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.294s
I20260709 13:04:38.399189 15235 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:38.444314 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.399385 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.399708 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.340521 19780 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:38.113054 20184 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.252s
I20260709 13:04:38.400362 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:38.403353 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 66741.4us
I20260709 13:04:38.404965 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.405767 15234 rpcz_store.cc:276] 0709 13:04:36.750065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.750259 (+   194us) service_pool.cc:224] Handling call
0709 13:04:36.772377 (+ 22118us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.772924 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.772969 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.831130 (+ 58161us) log.cc:844] Serialized 52 byte log entry
0709 13:04:37.218240 (+387110us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:37.218313 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.218406 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.990824 (+772418us) spinlock_profiling.cc:243] Waited 92.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:38.368755 (+377931us) raft_consensus.cc:1685] finished
0709 13:04:38.368778 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.369062 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30092,"raft.queue_time_us":3486,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":99,"spinlock_wait_cycles":213129600,"thread_start_us":148491,"threads_started":1,"wal-append.queue_time_us":496214}
I20260709 13:04:38.409575 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.409647 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.399880 20046 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.451373 15228 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
I20260709 13:04:38.414011 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:38.414391 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49762.7us
I20260709 13:04:38.452016 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.375882 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.416307 20017 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:38.391717 20107 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:38.455755 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.404798 20014 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.406751 20183 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:38.435935 15084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2742) took 2146 ms (client timeout 2134 ms). Trace:
W20260709 13:04:38.461804 15084 rpcz_store.cc:269] 0709 13:04:36.278252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.278666 (+   414us) service_pool.cc:224] Handling call
0709 13:04:37.695502 (+1416836us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b58008681d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9400aa 00007f0d3f9c2447 00007f0d3f9dafe7 00007f0d3f9dc7de 00007f0d3f9e827c 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c
0709 13:04:38.423995 (+728493us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400098dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:38.424130 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910544384}
I20260709 13:04:38.411551 19420 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:38.412036 19871 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:38.445235 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.467999 15231 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:38.445854 15230 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:38.446514 15156 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:38.453814 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.406586 20075 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.452610 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:38.387504 20220 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:38.457113 20212 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.461053 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.482949 15233 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:38.458807 20017 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:38.462683 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.422050 19942 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:04:38.441990 20026 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.441787 20188 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.428256 20173 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.450839 20003 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.478047 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.430397 19975 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.439049 20033 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:38.487979 15230 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:38.464622 20183 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:38.494949 15086 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:38.480250 20045 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.484645 19871 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:38.462306 20107 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.531002 15232 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 9.
W20260709 13:04:38.532768 20236 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:38.533669 19975 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:38.534283 20228 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:04:38.543931 20200 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.546307 20173 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.549598 20075 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.551560 20139 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.553607 20140 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.555379 19871 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.556262 20046 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.494072 20220 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:38.582653 20220 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.586458 20017 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.466702 20014 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:38.567019 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 157533us
I20260709 13:04:38.525231 20158 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:38.491080 20212 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.526621 20033 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.588789 20017 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.602849 20158 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:38.603572 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 160910us
I20260709 13:04:38.603730 20157 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.594851 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.610769 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.609267 20158 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:38.717460 20243 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:04:38.718225 19599 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.738442 20101 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:38.740731 20212 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.744190 20014 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.746044 19999 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:38.746625 19999 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:38.747720 19999 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.750268 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1068) took 2026 ms. Trace:
I20260709 13:04:38.750646 15160 rpcz_store.cc:276] 0709 13:04:36.724203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.724395 (+   192us) service_pool.cc:224] Handling call
0709 13:04:36.742357 (+ 17962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.080586 (+1338229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.750032 (+669446us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540033f754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:38.750208 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914177408}
I20260709 13:04:38.761917 20107 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:38.709653 20239 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:04:38.763494 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2325) took 1147 ms. Trace:
W20260709 13:04:38.763865 20153 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.000s
I20260709 13:04:38.769783 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.771418 20011 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:38.771876 20011 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:38.773098 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.774398 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:38.775029 15230 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:38.777621 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 347797us
I20260709 13:04:38.777730 20199 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:38.778544 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:38.779569 20199 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:38.772716 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:38.781064 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.781226 15237 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:38.782121 20204 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.782377 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.783077 15228 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:38.782784 20204 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.784390 15126 leader_election.cc:400] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14.
I20260709 13:04:38.776649 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:38.772701 20011 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.785579 20137 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:38.785988 20137 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:38.786247 19930 consensus_queue.cc:1175] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 7 index: 4 } }
I20260709 13:04:38.786532 20137 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:38.789042 20235 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:38.791716 20003 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:38.792725 20140 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:38.793164 20140 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:38.794064 20140 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:38.785925 20011 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:38.784924 15126 leader_election.cc:403] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:38.812098 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:38.810406 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 215616us
I20260709 13:04:38.800179 15230 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:38.813359 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:38.813067 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 360501us
I20260709 13:04:38.821811 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:38.763938 15161 rpcz_store.cc:276] 0709 13:04:37.616506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.616609 (+   103us) service_pool.cc:224] Handling call
0709 13:04:37.616879 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.681816 (+ 64937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.681844 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.682294 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.931082 (+248788us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:38.081550 (+150468us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.762844 (+681294us) spinlock_profiling.cc:243] Waited 885 us on lock 0x7b18003c3520. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:38.763037 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.763117 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.763162 (+    45us) raft_consensus.cc:1685] finished
0709 13:04:38.763268 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.763426 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114793,"prepare.queue_time_us":275881,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":766294,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":34072,"spinlock_wait_cycles":2034432,"thread_start_us":398494,"threads_started":2,"wal-append.queue_time_us":426130}
I20260709 13:04:38.832194 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.834458 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.834569 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.836171 15230 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:38.839787 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:38.872663 15844 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:38.874591 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.875829 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:38.876322 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.877662 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.878577 15087 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:38.880343 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:38.882506 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.884251 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:38.885121 15083 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:38.886374 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 275682us
W20260709 13:04:38.887252 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64900.6us
I20260709 13:04:38.887439 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:38.888180 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.883425 15844 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:38.807682 20140 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:38.894508 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.895704 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:38.768666 20033 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.906301 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:38.910315 15080 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:38.829651 19513 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:38.830452 20157 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14.
I20260709 13:04:38.915722 20045 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:38.915958 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 102681us
I20260709 13:04:38.916253 20045 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.916643 20045 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:38.917532 20025 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:38.918596 19935 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:38.920233 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:38.922115 20210 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.923437 20059 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:38.924826 20212 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:38.926709 20045 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.928889 20045 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:38.934634 20102 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:38.939525 15847 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:38.878283 20199 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 13:04:38.949853 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1252) took 1331 ms. Trace:
I20260709 13:04:38.872551 20026 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:38.877550 19999 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:38.953487 15160 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:38.955029 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.955816 15163 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:38.872236 20135 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:38.957212 20135 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:38.957695 20135 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:38.959250 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1201) took 2742 ms (client timeout 3122 ms). Trace:
W20260709 13:04:38.959755 15848 rpcz_store.cc:269] 0709 13:04:36.216690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.216947 (+   257us) service_pool.cc:224] Handling call
0709 13:04:38.958993 (+2742046us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540066f8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:38.959145 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3487260928}
I20260709 13:04:38.909463 19924 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:38.966514 20137 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.969396 20152 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:38.969966 20152 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:38.970541 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:38.970958 20045 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:38.971494 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:38.972210 20152 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:38.973927 15154 pending_rounds.cc:85] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 3
I20260709 13:04:38.974254 15154 pending_rounds.cc:107] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:04:38.974846 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.974956 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.974511 15154 raft_consensus.cc:2889] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.976955 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.950423 15845 rpcz_store.cc:276] 0709 13:04:37.618557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.618670 (+   113us) service_pool.cc:224] Handling call
0709 13:04:37.689350 (+ 70680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.957061 (+267711us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:38.187813 (+230752us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.524120 (+336307us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.526098 (+  1978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.526215 (+   117us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:38.526289 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.539218 (+ 12929us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b1800363d60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:38.792012 (+252794us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b1800363d60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:38.949551 (+157539us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:38.949611 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.949774 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73575,"prepare.queue_time_us":499253,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":99,"raft.queue_time_us":1858,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":30876,"spinlock_wait_cycles":644069120,"thread_start_us":531091,"threads_started":2,"wal-append.queue_time_us":342564}
I20260709 13:04:38.936542 20102 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:38.890237 20139 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:38.888797 20175 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.998155 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:38.998502 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.001616 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:39.003499 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:39.004503 15158 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:38.897550 20217 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:38.898545 20248 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:39.007694 15161 raft_consensus.cc:2368] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:39.005477 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.012321 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:39.023387 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.026541 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.027041 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:39.028900 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.031713 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.033779 15161 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:39.035235 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.036895 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.038865 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.039834 15084 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:39.041555 15083 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:39.042819 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.065722 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.046787 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:39.048496 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 162394us
I20260709 13:04:39.027415 15163 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:38.992246 20058 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.543s	user 0.001s	sys 0.009s
I20260709 13:04:39.055570 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.057518 20033 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.078024 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.078528 15087 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:39.078918 15237 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 8.
I20260709 13:04:39.059600 20246 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:39.061869 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:38.990422 20071 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:39.064546 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.990034 20022 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:39.071658 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:39.072714 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:38.986271 20011 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.066953 15088 raft_consensus.cc:2376] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13. Re-sending same reply.
I20260709 13:04:39.102197 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.015455 20153 raft_consensus.cc:476] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:39.017655 20102 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:38.993692 20139 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:39.068190 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:39.097924 19780 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.134s	sys 0.198s
I20260709 13:04:39.115527 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.077140 15083 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:38.991343 20212 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 13:04:39.150578 15230 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:39.131121 15160 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:39.137861 15228 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:39.161955 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.162827 15234 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:39.164379 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.165462 15228 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:39.092581 20071 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:39.085405 20218 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:39.101490 20022 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:39.154174 19780 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.134s	sys 0.199s
W20260709 13:04:39.209925 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 303702us
I20260709 13:04:39.154393 20139 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:39.072942 20158 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.085150 20058 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:39.154808 20137 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:39.170017 20183 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.003s
I20260709 13:04:39.246371 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:39.218232 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65435.4us
I20260709 13:04:39.128726 20011 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.100195 20246 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.315223 15081 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:04:39.296123 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 182474us
W20260709 13:04:39.218475 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 150392us
I20260709 13:04:39.249497 20102 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:39.430054 20265 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
W20260709 13:04:39.431061 20254 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:04:39.431417 20065 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.214591 20022 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:39.267158 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.270069 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:39.432693 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:39.433601 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 163587us
I20260709 13:04:39.271999 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:39.434944 15237 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:39.276083 15229 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:39.090241 20033 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.277155 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:39.169338 20026 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.287117 20193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.041s	sys 0.083s
I20260709 13:04:39.405189 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.448484 15156 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:39.203342 20071 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:39.425387 20255 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.167s	user 0.000s	sys 0.004s
I20260709 13:04:39.436081 15160 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
W20260709 13:04:39.380476 20249 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:04:39.368379 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:39.458303 15846 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:39.434373 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:39.463851 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:39.379091 20253 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:04:39.366379 20209 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:39.232700 20218 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.381220 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.277137 20017 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:39.360420 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1264) took 2135 ms. Trace:
I20260709 13:04:39.300390 20199 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:39.273350 19780 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:39.233248 19924 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:39.362110 20058 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:39.471060 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.515077 20218 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.515992 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.517256 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.472059 15841 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:39.526718 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.281828 20183 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.475179 20045 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:39.335693 20251 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:39.449748 20193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.052s	sys 0.090s
W20260709 13:04:39.509627 20220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.016s	sys 0.027s
I20260709 13:04:39.497213 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.512514 20246 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.511981 19780 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:39.513700 20269 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:39.468986 20209 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:39.442924 20026 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.510001 20188 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:39.479756 20017 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:39.465533 20135 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.475845 20248 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:04:39.527690 19924 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:39.541565 20220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.016s	sys 0.027s
I20260709 13:04:39.536690 20045 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:39.548734 20014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.022s	sys 0.057s
I20260709 13:04:39.562249 20017 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:39.554396 20158 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.548295 20188 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:39.597205 20014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.023s	sys 0.057s
I20260709 13:04:39.597880 20045 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:39.610219 20188 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:39.643118 20183 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.645385 20135 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.650020 20139 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:39.677310 20140 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.000s
I20260709 13:04:39.684363 20139 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:39.437021 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:39.763432 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 326244us
W20260709 13:04:39.298696 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 410594us
I20260709 13:04:39.436586 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:39.770081 15082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.012s
W20260709 13:04:39.770768 15082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.000s	sys 0.012s
W20260709 13:04:39.771685 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 335043us
I20260709 13:04:39.775511 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:39.768605 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:39.450670 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.785723 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 485430us
W20260709 13:04:39.389163 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.790141 15161 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:39.790745 15156 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:39.241953 15158 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:39.798723 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:39.799978 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:39.807046 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:39.461956 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:39.473098 15839 rpcz_store.cc:276] 0709 13:04:37.225221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.227529 (+  2308us) service_pool.cc:224] Handling call
0709 13:04:37.227970 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.057083 (+829113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.360040 (+1302957us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:39.360309 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793439488}
I20260709 13:04:39.553154 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.915777 20026 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.921831 20033 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.928937 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:39.931418 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.938197 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.939680 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.940613 15841 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:39.940986 20158 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:39.948179 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 146771us
I20260709 13:04:39.953788 20185 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:39.955969 20235 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:39.957818 20183 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:39.958140 19596 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:39.958534 19596 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:39.960779 15127 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:39.962178 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:39.965256 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.960822 20188 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:39.862944 15844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.006s	sys 0.018s
W20260709 13:04:39.967244 15844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.007s	sys 0.018s
I20260709 13:04:39.967562 20102 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:39.968072 20102 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.968426 20102 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:39.879101 19813 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:39.882556 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 75800.8us
I20260709 13:04:39.969099 19813 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.969552 19813 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:39.971938 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:39.973327 20212 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:39.974335 15156 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:39.974574 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.975252 20183 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.975715 19930 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:39.976121 20183 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.978011 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:39.979153 15080 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:39.979378 20183 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.976402 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:39.890151 20276 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:39.893191 20193 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:39.893334 20220 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 13:04:39.883651 15201 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.014s	sys 0.005s
W20260709 13:04:39.607941 19999 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.003s
I20260709 13:04:39.908957 20199 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:39.910077 15845 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.012s	sys 0.034s
I20260709 13:04:39.911883 19924 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:39.912086 20251 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:39.966600 19818 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:39.955132 20184 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 13:04:39.862571 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77973.8us
I20260709 13:04:39.984092 15159 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:04:39.999033 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:40.000463 15844 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:04:39.960088 19596 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:39.956923 20235 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:40.003613 15201 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.015s	sys 0.005s
I20260709 13:04:39.985549 20276 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:39.991608 20188 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:40.007171 15845 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.013s	sys 0.035s
I20260709 13:04:40.009868 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:39.893256 20011 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:40.022535 20071 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.004s
I20260709 13:04:40.036159 20066 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.036655 20066 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:40.037954 20066 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.039609 20066 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.034425 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.021334 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.044154 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.025902 20276 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:40.013206 20199 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.006366 20251 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:39.609750 20209 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.033931 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.054034 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.061637 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:40.067584 20022 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.617s	user 0.000s	sys 0.012s
I20260709 13:04:40.068076 20022 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.068950 15082 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
W20260709 13:04:40.059034 15084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.013s	sys 0.009s
W20260709 13:04:40.075692 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 849845us
W20260709 13:04:40.079420 15084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.014s	sys 0.009s
I20260709 13:04:40.078473 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.086967 20045 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.093298 15842 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:40.094637 20045 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.094324 20251 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.076143 20184 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:40.104925 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:40.105484 20238 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.514s	user 0.000s	sys 0.002s
I20260709 13:04:40.109586 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1230) took 1065 ms. Trace:
I20260709 13:04:40.111325 15846 raft_consensus.cc:2368] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 9. Current term is 10.
I20260709 13:04:40.113482 20209 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:40.121064 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45665.2us
W20260709 13:04:40.124233 20252 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.956s	user 0.005s	sys 0.022s
I20260709 13:04:40.125020 20188 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:40.125634 20188 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:40.125690 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:40.126331 20188 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:40.129344 20140 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:40.130273 15841 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:40.131184 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1271) took 1013 ms. Trace:
I20260709 13:04:40.131604 20140 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:40.131673 15841 rpcz_store.cc:276] 0709 13:04:39.117681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.090094 (+972413us) service_pool.cc:224] Handling call
0709 13:04:40.129690 (+ 39596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.130774 (+  1084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.131088 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:40.134733 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2317) took 1721 ms (client timeout 2073 ms). Trace:
W20260709 13:04:40.135325 15232 rpcz_store.cc:269] 0709 13:04:38.412882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.454937 (+ 42055us) service_pool.cc:224] Handling call
0709 13:04:40.134269 (+1679332us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b5400295754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:40.134490 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98721152}
I20260709 13:04:40.137565 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1278) took 1018 ms. Trace:
W20260709 13:04:40.093043 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 121541us
I20260709 13:04:40.138023 15844 rpcz_store.cc:276] 0709 13:04:39.119221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.135852 (+1016631us) service_pool.cc:224] Handling call
0709 13:04:40.136460 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.136989 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.137023 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.137095 (+    72us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.137288 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.137368 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.137503 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.139901 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:40.140272 15844 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:40.131781 20136 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:40.141415 20136 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.141599 20248 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:40.110102 15084 rpcz_store.cc:276] 0709 13:04:39.044660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.067684 (+ 23024us) service_pool.cc:224] Handling call
0709 13:04:39.068055 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.068442 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.068484 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.069034 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.109021 (+1039987us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:40.109097 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.109238 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.109306 (+    68us) raft_consensus.cc:1685] finished
0709 13:04:40.109317 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.109507 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50434,"raft.queue_time_us":5144,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":13800,"thread_start_us":1014556,"threads_started":1,"wal-append.queue_time_us":935068}
I20260709 13:04:40.143687 20207 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.147665 20207 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:40.148332 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 137760us
I20260709 13:04:40.149821 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.150159 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:40.104065 20017 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.151916 20053 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.152976 20053 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.154166 20135 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.155041 20135 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.155911 20053 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.143160 20248 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:40.142989 20136 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.158679 20246 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:40.145999 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.063943 20193 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.206565 20193 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.173345 20140 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:40.207628 20140 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.209028 20140 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.210863 20140 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:40.211886 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1269) took 1094 ms. Trace:
I20260709 13:04:40.216437 20022 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.218423 15233 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:40.220515 15159 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:40.221647 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:40.222270 15159 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:40.222622 20204 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.173233 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.169052 20248 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:40.226226 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:40.226178 20248 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.227959 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:40.175499 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.176286 15232 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:40.181159 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.181258 20199 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.182682 15163 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:40.050535 20071 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.185156 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.154109 20017 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.186157 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.186890 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:40.187604 15082 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:40.190241 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:40.190575 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.193426 15085 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:40.193933 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1084) took 1142 ms. Trace:
I20260709 13:04:40.194988 20207 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.195528 20235 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.170135 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1280) took 1015 ms. Trace:
I20260709 13:04:40.197484 20135 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.198168 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.198431 15229 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:40.199366 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.199561 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:40.199815 20136 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.200016 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:40.172966 20045 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.203034 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:40.204221 15202 connection.cc:256] RPC call timeout handler was delayed by -1.1403332140000657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2405s for a response
I20260709 13:04:40.228817 15158 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:40.229960 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54585.6us
I20260709 13:04:40.256553 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:40.232151 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:40.212394 15846 rpcz_store.cc:276] 0709 13:04:39.117175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.546257 (+429082us) service_pool.cc:224] Handling call
0709 13:04:40.211611 (+665354us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540001a4d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:40.211821 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393562496}
I20260709 13:04:40.236260 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:40.250972 15844 rpcz_store.cc:276] 0709 13:04:39.130308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.139313 (+1009005us) service_pool.cc:224] Handling call
0709 13:04:40.139820 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.140737 (+   917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.145059 (+  4322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.234998 20183 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.280929 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.263036 20045 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:40.284073 15848 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:40.285202 15160 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:40.286404 15088 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:04:40.288367 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56439.1us
I20260709 13:04:40.282308 20207 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.289234 20053 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:40.273588 20246 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:40.307828 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.293097 20153 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.294523 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:40.295166 20135 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:40.266592 20102 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.231611 20139 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:04:40.298964 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2139) took 1417 ms. Trace:
I20260709 13:04:40.307071 15084 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:40.299531 20208 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.282908 20235 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.301976 15158 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:40.302543 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.280328 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.310344 20053 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.302891 14981 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 7 to 9, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 9 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:40.310763 20207 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.311748 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:40.312136 20071 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.312443 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:40.315757 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1286) took 1062 ms. Trace:
I20260709 13:04:40.319370 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.237737 15155 rpcz_store.cc:276] 0709 13:04:38.946365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.976298 (+ 29933us) service_pool.cc:224] Handling call
0709 13:04:38.980332 (+  4034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.980583 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:39.465710 (+485127us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.030685 (+564975us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.048261 (+ 17576us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.072727 (+ 24466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.076037 (+  3310us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.089114 (+ 13077us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6538,"prepare.queue_time_us":376708,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":13195,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":17530,"thread_start_us":955228,"threads_started":2,"wal-append.queue_time_us":455275}
I20260709 13:04:40.334646 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.320319 20246 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:40.325587 15081 rpcz_store.cc:276] 0709 13:04:38.881134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.881252 (+   118us) service_pool.cc:224] Handling call
0709 13:04:38.883095 (+  1843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.427861 (+544766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.298431 (+870570us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400190354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:40.298774 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993001728}
I20260709 13:04:40.334337 15842 rpcz_store.cc:276] 0709 13:04:39.253740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.932793 (+679053us) service_pool.cc:224] Handling call
0709 13:04:40.315547 (+382754us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540074e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:40.315694 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292194048}
I20260709 13:04:40.375016 15156 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:40.327499 20208 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:40.304973 19813 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:40.382114 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.380262 15079 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:40.382848 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:40.333307 20135 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.337949 20246 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:40.340783 20066 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.385978 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:40.391923 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 484024us
I20260709 13:04:40.391278 15842 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:40.399737 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.404826 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.405457 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:40.410315 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 230392us
I20260709 13:04:40.412465 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:40.380741 20188 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:40.405295 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 122409us
W20260709 13:04:40.418529 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 124211us
I20260709 13:04:40.382526 20235 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.419625 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:40.421818 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1226) took 1567 ms. Trace:
I20260709 13:04:40.422690 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2750) took 2785 ms. Trace:
I20260709 13:04:40.415252 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:40.434253 20135 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.441116 15235 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:40.441577 20053 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.447166 20102 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.450304 20025 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.450749 20025 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:40.451756 20025 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.453281 20208 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.454713 20208 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.455734 19596 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:40.455962 19813 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.458853 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.465783 20207 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.405151 20153 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:40.467633 20235 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.468986 20153 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.469767 15236 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:40.470675 15154 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:40.471693 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:40.472312 15846 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.474150 20188 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.474419 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.427309 15086 rpcz_store.cc:276] 0709 13:04:37.603728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.609755 (+  6027us) service_pool.cc:224] Handling call
0709 13:04:37.610007 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.518092 (+908085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.327194 (+1809102us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b5400349254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:40.389369 (+ 62175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034607360}
I20260709 13:04:40.477715 15159 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:04:40.464565 20274 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.855s	user 0.017s	sys 0.000s
W20260709 13:04:40.479434 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46686.6us
I20260709 13:04:40.480131 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.480576 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2140) took 1595 ms. Trace:
I20260709 13:04:40.429891 20209 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:40.481777 15844 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:40.481353 20045 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.482635 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.486842 15158 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 8.
I20260709 13:04:40.485670 15085 rpcz_store.cc:276] 0709 13:04:38.885398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.885609 (+   211us) service_pool.cc:224] Handling call
0709 13:04:38.885932 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.239520 (+1353588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.475968 (+236448us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540039fdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:40.480507 (+  4539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950496128}
I20260709 13:04:40.483098 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.483268 20217 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:40.491891 15846 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:40.494313 15155 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:40.494132 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.422266 15845 rpcz_store.cc:276] 0709 13:04:38.848103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.979273 (+131170us) service_pool.cc:224] Handling call
0709 13:04:39.020075 (+ 40802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.020356 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:40.078006 (+1057650us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.378773 (+300767us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.387504 (+  8731us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.406505 (+ 19001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.409772 (+  3267us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.415905 (+  6133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53683,"prepare.queue_time_us":71037,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":44843,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":102,"thread_start_us":1201605,"threads_started":2,"wal-append.queue_time_us":325029}
I20260709 13:04:40.493943 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.503379 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:40.508201 15160 raft_consensus.cc:2410] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:40.512358 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.499393 20025 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.452685 20276 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.514976 15157 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:04:40.535856 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 107764us
W20260709 13:04:40.539161 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.532982 20235 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:40.540422 20235 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.542097 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.542971 20235 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.543787 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.545064 20276 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.549006 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1219) took 1645 ms. Trace:
I20260709 13:04:40.549655 20302 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:40.549530 15840 rpcz_store.cc:276] 0709 13:04:38.845564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.845701 (+   137us) service_pool.cc:224] Handling call
0709 13:04:38.902322 (+ 56621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.993566 (+ 91244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.993597 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.014065 (+ 20468us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.209838 (+1195773us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b180041a4c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:40.410064 (+200226us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.416004 (+  5940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.431519 (+ 15515us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.476916 (+ 45397us) raft_consensus.cc:1685] finished
0709 13:04:40.476936 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.490589 (+ 13653us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121849,"raft.queue_time_us":91049,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":16963,"spinlock_wait_cycles":17516416,"thread_start_us":179596,"threads_started":1,"wal-append.queue_time_us":1407350}
I20260709 13:04:40.551154 20302 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:40.552229 20302 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:40.552928 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.553041 20053 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.553593 20053 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:40.554674 20053 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.556221 20053 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.556272 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.557508 14981 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 11 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.558200 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.512084 20217 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.502925 20209 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:40.845350 20140 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.010s
I20260709 13:04:40.846828 20209 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:40.594743 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 51116.9us
W20260709 13:04:40.631102 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 220884us
W20260709 13:04:40.659030 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 171080us
I20260709 13:04:40.601382 20045 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.676653 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1052) took 1310 ms. Trace:
W20260709 13:04:40.813252 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.559046 19997 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:40.514710 20153 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:40.517557 20183 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.855795 20066 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:40.826437 15231 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:40.857161 20066 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:40.857354 20153 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.858814 20251 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:40.859938 20153 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:40.828857 20302 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:40.829047 20248 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:40.860338 19780 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:40.860812 19780 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:40.860810 20248 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.853207 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:40.829732 20285 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:40.863574 20248 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.863811 20285 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:40.831094 20071 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:40.820782 20184 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.723s	user 0.023s	sys 0.001s
W20260709 13:04:40.837378 20136 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.572s	user 0.014s	sys 0.003s
I20260709 13:04:40.840183 20276 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:40.840965 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 310992us
I20260709 13:04:40.840876 20178 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:40.871793 20184 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:40.877295 20199 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:40.877772 20199 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.877820 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:40.878755 15231 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:04:40.879038 20199 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.879702 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:40.881294 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.883062 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.883634 20212 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:40.885017 20276 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.885401 20217 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.886992 20303 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:40.887521 20303 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:40.887754 20276 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.888020 20303 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:40.889889 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:40.829854 19596 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:40.865195 19997 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:40.864171 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:40.864300 20285 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:40.869274 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:40.872646 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:40.869803 20140 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.851755 15234 rpcz_store.cc:276] 0709 13:04:39.229233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.229538 (+   305us) service_pool.cc:224] Handling call
0709 13:04:39.284363 (+ 54825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.284949 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.284988 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.318043 (+ 33055us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.024768 (+706725us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:40.330759 (+305991us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.529803 (+199044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.539493 (+  9690us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.539556 (+    63us) raft_consensus.cc:1685] finished
0709 13:04:40.539575 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.539739 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":364515,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":35435,"raft.queue_time_us":166708,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":21364,"thread_start_us":888020,"threads_started":3,"wal-append.queue_time_us":483929}
I20260709 13:04:40.905257 15162 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:04:40.874694 20308 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:40.901533 14981 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 11, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:40.907824 20285 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.836532 20246 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.857858 20066 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:40.862160 19780 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:40.876240 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:40.914552 20066 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:40.897822 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.827430 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.916213 20066 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:40.916545 20251 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.917351 20285 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:40.917789 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 95566.2us
I20260709 13:04:40.919049 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:40.890273 15231 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:40.903936 19596 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:40.910678 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46574.9us
I20260709 13:04:40.908778 20053 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:40.914865 19780 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:40.919983 20251 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:40.920749 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48196.8us
I20260709 13:04:40.871047 20208 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.922211 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:40.867612 20207 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:40.880630 20199 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:40.944790 20246 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.945528 20135 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:40.941491 14981 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 5 to 11, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 11 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:40.946254 20135 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.928967 20153 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:40.947283 14981 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:40.929189 20136 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:40.934342 20276 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:40.936259 20212 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:40.965343 20135 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:40.973769 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 120738us
I20260709 13:04:40.939442 20207 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:40.982969 14981 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 8 to 10, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:40.944790 19596 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:40.981451 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:40.982368 15840 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:04:40.983083 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 248201us
W20260709 13:04:40.979826 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.950273 20140 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.956229 20136 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.953263 20053 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:40.991886 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:40.996567 15842 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:40.993767 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:40.995345 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73236.2us
I20260709 13:04:40.998788 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.001230 20291 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.001967 20291 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:40.975346 20212 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.998575 20136 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.004331 20291 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:40.961104 20102 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.005247 20102 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.007231 20102 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:41.010409 15813 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.611s (SENT)
I20260709 13:04:41.003998 20212 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:40.998946 20053 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.017926 20053 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:41.018488 15162 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:41.019620 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.020071 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:41.020426 20071 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:41.020913 15161 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:41.021081 20071 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.023241 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:41.024060 15155 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:41.025013 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43662.5us
I20260709 13:04:40.967891 20208 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.025581 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:41.028898 20246 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:41.031049 15160 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:41.031333 15054 leader_election.cc:400] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:41.031769 15054 leader_election.cc:403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:41.025772 20045 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:41.033123 20246 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.033640 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.034991 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.035691 20183 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.037325 15154 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:41.038929 15157 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:04:41.039312 15157 pending_rounds.cc:85] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 8
I20260709 13:04:41.039562 15157 pending_rounds.cc:107] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:41.039896 15157 raft_consensus.cc:2889] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:41.012418 15813 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.638s (SENT)
I20260709 13:04:40.963452 20276 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.031371 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.012450 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:41.031139 20251 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:41.048256 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.049175 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.051674 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.053604 15162 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:41.055501 15160 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:04:41.056926 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.058588 14981 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:41.059585 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.060211 15839 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.045964 20251 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.060076 14981 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:41.065637 15161 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:41.081496 14982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 262) took 38885 ms (client timeout 15000 ms). Trace:
I20260709 13:04:41.065616 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.079259 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.105422 15839 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:41.106961 15155 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:41.110967 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.111248 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.111979 15842 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:41.112535 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.067813 20136 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.113899 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.114603 15845 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:41.102535 14982 rpcz_store.cc:269] 0709 13:04:02.196250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.196639 (+   389us) service_pool.cc:224] Handling call
0709 13:04:41.054821 (+38858182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:31.709694 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:31.749206 (+ 39512us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:31.749270 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:04:31.749294 (+    24us) tablet.cc:662] Decoding operations
0709 13:04:31.750845 (+  1551us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:31.750986 (+   141us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:31.751015 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:04:31.752259 (+  1244us) tablet.cc:701] Row locks acquired
0709 13:04:31.752314 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:04:31.752506 (+   192us) write_op.cc:270] Start()
0709 13:04:31.752657 (+   151us) write_op.cc:276] Timestamp: P: 1783602271752432 usec, L: 0
0709 13:04:31.752673 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:04:31.915393 (+162720us) log.cc:844] Serialized 5940 byte log entry
0709 13:04:32.184560 (+269167us) op_driver.cc:464] REPLICATION: finished
0709 13:04:32.451123 (+266563us) write_op.cc:301] APPLY: starting
0709 13:04:32.451262 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.805806 (+354544us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.805940 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.934195 (+128255us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.934530 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:04:32.934581 (+    51us) write_op.cc:312] APPLY: finished
0709 13:04:32.934934 (+   353us) log.cc:844] Serialized 139 byte log entry
0709 13:04:32.936047 (+  1113us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.936271 (+   224us) write_op.cc:454] Released schema lock
0709 13:04:32.936349 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.940461 (+  4112us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.940506 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":871326,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":266216,"apply.run_cpu_time_us":23619,"apply.run_wall_time_us":489286,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":209237,"prepare.run_cpu_time_us":17048,"prepare.run_wall_time_us":354208,"raft.queue_time_us":85694,"raft.run_cpu_time_us":69275,"raft.run_wall_time_us":256447,"replication_time_us":431724,"thread_start_us":550079,"threads_started":4,"wal-append.queue_time_us":160866,"wal-append.run_cpu_time_us":27458,"wal-append.run_wall_time_us":1862480}]]}
I20260709 13:04:41.115329 20136 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.046001 19813 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:41.081141 20188 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:41.047796 20276 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:41.116477 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.119328 15846 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:41.119395 15848 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:41.115609 15845 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:41.117036 15844 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:41.120500 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:41.117938 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.116308 20188 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:41.121687 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.103053 15841 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:41.123176 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.051338 20184 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:41.128274 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.056505 20178 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.129844 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.130004 20178 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.132431 20178 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.066112 20071 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:41.117475 20251 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.081122 20209 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.144713 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.124752 14981 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.073966 20208 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:40.991981 19997 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.070964 20053 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:41.157217 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 144862us
I20260709 13:04:41.128924 19596 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:41.153815 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.156277 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.160302 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:41.069502 20217 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:04:41.125485 20302 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.151793 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.172595 15846 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:41.158892 14981 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:41.164538 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.175980 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 163648us
I20260709 13:04:41.177059 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.120680 20303 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.070830 20212 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.178864 15841 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:41.169610 15842 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:41.065155 20207 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.217146 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.192668 20184 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:41.219312 20208 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:41.219640 20184 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:41.193161 19997 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:41.193703 20212 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.201284 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.225519 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.190310 20188 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.188479 20102 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.204924 15235 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:41.205761 20309 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:41.205929 20302 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.208076 20303 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.208619 15232 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:41.213990 20053 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.191241 14981 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 10 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.224094 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.225675 15233 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:41.159523 20209 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.240763 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.238144 20188 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.245298 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:41.248122 15813 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.837s (SENT)
I20260709 13:04:41.249951 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.250726 15840 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.239992 15844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:41.247300 15847 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:41.256090 15843 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:41.233946 20102 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.259253 20102 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:41.283970 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.284765 14981 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:41.266995 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 417734us
I20260709 13:04:41.304456 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.307111 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.316236 19780 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:41.322321 20246 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:41.329250 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:41.331715 20251 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.335016 20209 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.287859 20071 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 13:04:41.342229 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.342929 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.343019 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2333) took 1064 ms. Trace:
I20260709 13:04:41.343915 20289 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:41.344681 20289 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:41.345510 20045 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:41.345772 20289 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.346477 20045 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.346920 20045 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.348973 20102 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.349995 20025 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.351222 20285 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:41.351419 20300 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:41.351614 20285 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:41.352177 20300 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.367095 15840 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:41.368484 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 111957us
I20260709 13:04:41.369462 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:41.381757 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.382397 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:41.310614 20217 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.381444 20071 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:41.349550 20102 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.384874 14981 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:41.396595 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.397768 15232 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:41.343621 15229 rpcz_store.cc:276] 0709 13:04:40.278626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.278801 (+   175us) service_pool.cc:224] Handling call
0709 13:04:40.321233 (+ 42432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.411406 (+ 90173us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:40.658068 (+246662us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.248239 (+590171us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.332157 (+ 83918us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:41.342262 (+ 10105us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b18003fbc20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:41.342516 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.342584 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:41.342659 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.342779 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.342944 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10313,"prepare.queue_time_us":34057,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":105214,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10141,"spinlock_wait_cycles":52514176,"thread_start_us":593678,"threads_started":2,"wal-append.queue_time_us":483915}
I20260709 13:04:41.371474 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:41.352674 20285 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:41.406917 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 153108us
W20260709 13:04:41.421862 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50470.3us
W20260709 13:04:41.422976 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 93837.2us
I20260709 13:04:41.395327 20199 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.423875 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.399204 20140 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:41.420462 14981 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.366189 20291 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:04:41.369908 20276 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.387588 20209 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:04:41.434803 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 191207us
I20260709 13:04:41.401382 20071 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:41.407083 20309 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:41.309937 20184 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.436153 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.437080 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:41.438431 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 402101us
I20260709 13:04:41.440616 14981 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.428018 19596 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:41.395969 20217 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.450917 14981 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.431887 20140 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.439514 20285 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:41.459025 20204 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:41.491750 20204 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:41.492913 20204 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.456697 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:41.501950 15086 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
W20260709 13:04:41.503819 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.018s	sys 0.027s
I20260709 13:04:41.463446 20045 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.465833 20302 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.465379 14981 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 9, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 9 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:41.466703 20217 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.456307 20309 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:41.468283 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.490586 20140 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.469955 15084 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:41.532546 15844 raft_consensus.cc:606] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:41.470441 20316 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:41.471669 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:41.472146 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.434590 20199 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.473067 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:41.506063 20291 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:41.475549 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.479442 15088 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:41.480239 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:41.480476 20246 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:41.470491 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:41.482720 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:41.455976 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:41.564316 15079 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:41.461122 20276 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:41.506289 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.018s	sys 0.027s
I20260709 13:04:41.438000 20208 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:41.503685 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:41.566689 15085 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:04:41.513157 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.465436 20251 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.446444 20153 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:04:41.555402 15087 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:41.559271 15082 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.473933 20314 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:41.531152 14981 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 9, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.507800 20207 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:41.562139 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 93990.8us
I20260709 13:04:41.564515 15080 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:04:41.575115 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 104697us
I20260709 13:04:41.595456 20053 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:41.582540 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 692686us
I20260709 13:04:41.587577 20302 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.604238 14981 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.609788 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.610755 20322 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:41.611284 20045 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.562805 20291 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:41.613807 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 138349us
I20260709 13:04:41.529192 20071 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:41.614571 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.574247 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 147916us
W20260709 13:04:41.616524 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 194833us
W20260709 13:04:41.622198 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.570920 20208 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:41.573577 20316 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:41.614944 14981 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 9, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.613229 20308 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:41.629884 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:41.613466 19596 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:41.643332 15083 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:41.644088 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:41.645648 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 88423.7us
W20260709 13:04:41.618839 20312 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.303s	user 0.003s	sys 0.000s
I20260709 13:04:41.646790 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:41.583812 20246 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:41.669194 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54091us
W20260709 13:04:41.639297 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.670832 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.631986 20208 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:41.639479 20308 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.681464 15201 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.687984 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.666038 20276 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:41.691725 15843 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.665172 20322 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:41.701134 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.701867 15232 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.690186 20066 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:41.705530 20066 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:41.706002 20066 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.706216 20291 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.707652 20291 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:41.709654 20325 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:04:41.710688 15079 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 9.
I20260709 13:04:41.714592 14981 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 8 to 9, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.716426 15843 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:41.716519 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80581.3us
I20260709 13:04:41.720557 15233 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:41.720510 15052 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:41.714461 15235 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:04:41.725844 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 44467.9us
I20260709 13:04:41.726608 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.729689 20321 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:41.730865 20321 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:41.731375 20321 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.731731 20321 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.732003 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:41.733024 19780 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:41.733980 19813 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:41.734396 19813 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:41.734652 20183 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:41.735062 20183 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.734902 20066 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.735550 20183 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:41.730871 20251 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:41.738967 20324 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:04:41.720510 20102 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.692047 20178 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:41.743575 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.689965 20308 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.694307 20335 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:41.749663 20199 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.751246 15231 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:04:41.724778 20204 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.724272 20314 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:04:41.783989 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:41.753538 20318 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:04:41.691561 20217 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:04:41.741400 20251 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:41.756747 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.756969 19596 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.865381 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.846396 20204 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.862746 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.869721 15845 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:41.869995 20204 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:41.846244 20320 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:04:41.748207 20322 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.768553 15082 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:41.674680 20309 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.780117 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.881327 15230 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:41.777020 14981 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 9, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 9 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:41.881616 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.846274 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.743934 20178 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:41.851189 15848 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:41.846469 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.851281 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.846138 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.887619 15236 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:41.829759 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 144999us
I20260709 13:04:41.890774 20178 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:41.893878 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:41.894623 15156 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:41.901158 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.902128 15841 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:41.902835 20317 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:41.903293 20317 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:41.903750 20302 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:41.904224 20317 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.905467 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.830699 19780 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:41.906136 20337 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:41.906404 19780 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:41.906909 20337 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.907727 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.909226 15235 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:41.908766 14981 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 9, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 9 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.910450 15156 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:41.910389 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:41.841681 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.648s (SENT) [suppressed 36 similar messages]
I20260709 13:04:41.907955 20178 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:41.911585 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 69966.6us
I20260709 13:04:41.911967 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.878225 15845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e777c555e1b4153913b5e6459164454"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:41.913194 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:41.913466 15845 raft_consensus.cc:606] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:41.913909 15845 raft_consensus.cc:614] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:41.879129 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.914935 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.915423 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.915612 15845 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:41.880492 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:41.742208 20303 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:41.872622 20324 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:41.846026 20330 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:04:41.917411 20324 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:41.851408 20335 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.918388 20324 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.919780 20324 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:41.785199 19997 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.002s
I20260709 13:04:41.920544 15233 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:41.910295 14981 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.921356 15159 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:41.728386 20276 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.921763 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.922542 15233 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:41.925184 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.926196 15156 raft_consensus.cc:2410] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:41.888921 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:41.927093 15230 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:04:41.802790 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.928404 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:41.928761 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.929589 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.927222 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:41.930433 15842 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:41.906049 20317 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.934850 15232 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:41.936151 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.941819 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.942008 15162 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:41.941745 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.941951 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.947409 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:41.948696 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 59917.4us
I20260709 13:04:41.953521 15157 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:04:41.922855 20335 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:41.929610 14981 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: config changed from index 3 to 6, term changed from 8 to 10, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to <none>, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193), VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194), VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195), VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:41.956492 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:41.956048 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.953951 15157 pending_rounds.cc:85] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:41.956171 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.957615 15157 pending_rounds.cc:107] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:04:41.913162 20178 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.957875 15157 raft_consensus.cc:2889] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:41.959460 15157 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:41.959388 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.960328 20066 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:41.962107 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.963416 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:41.963682 15233 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:41.964290 15161 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:41.966550 20140 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:41.968852 15840 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:41.968551 20208 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:41.971005 20208 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:41.851936 20332 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:04:41.978654 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.980013 15231 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 8.
I20260709 13:04:41.981215 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.981935 15235 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:41.982873 15154 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:41.979483 15163 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:41.985836 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.987294 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.989040 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:41.990023 15232 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:04:41.993513 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:41.994573 15156 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:41.996737 15236 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
W20260709 13:04:41.998960 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49516.4us
I20260709 13:04:42.001235 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:42.002498 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:42.003404 20314 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.006022 20066 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.007153 15082 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:42.007335 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.009260 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.009462 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.007514 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.010152 20302 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:42.006886 20066 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.013885 15158 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:41.999688 15161 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.020069 15844 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:41.998250 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:41.998493 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 350371us
I20260709 13:04:42.028005 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:42.028661 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101522us
I20260709 13:04:42.029675 15236 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:04:42.029573 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:42.028746 14981 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 8 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:42.031565 15230 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:42.032420 15229 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:42.031560 14981 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 4 to 8, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 8 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:42.026041 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:41.957188 20303 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:42.041786 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 309556us
I20260709 13:04:42.042325 15156 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:42.044147 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.044284 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:42.045292 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:42.045473 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.045680 15233 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:41.953804 20342 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:42.042817 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:42.047158 15230 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:42.049217 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.050447 15232 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:41.995031 19997 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.993363 20184 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:42.054574 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:42.050407 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.067092 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.068056 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.069216 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.068099 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1063) took 1220 ms. Trace:
I20260709 13:04:42.070962 15236 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:42.070076 15228 rpcz_store.cc:276] 0709 13:04:40.555005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.557480 (+  2475us) service_pool.cc:224] Handling call
0709 13:04:40.822326 (+264846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.231118 (+408792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.231148 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.244537 (+ 13389us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.533080 (+288543us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:41.686936 (+153856us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.775420 (+ 88484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.775503 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.775552 (+    49us) raft_consensus.cc:1685] finished
0709 13:04:41.775566 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.775701 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38000,"prepare.queue_time_us":158135,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":80113,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":22527,"thread_start_us":687286,"threads_started":3,"wal-append.queue_time_us":438816}
I20260709 13:04:42.072448 15236 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:42.073058 20140 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.073861 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:42.074455 15157 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:42.050973 20178 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:04:42.077863 20066 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.077109 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:42.046154 20291 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:42.078831 15158 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:42.079130 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 76735.5us
I20260709 13:04:42.081089 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.082442 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.042191 20303 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:41.926411 20316 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.078552 20066 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:42.106889 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 118900us
W20260709 13:04:42.109543 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93789.3us
I20260709 13:04:42.041829 20053 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:42.053946 20338 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:42.102471 15081 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:42.113824 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:42.115150 15237 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:04:42.115612 15237 pending_rounds.cc:85] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 9
I20260709 13:04:42.115854 15237 pending_rounds.cc:107] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:04:42.116155 15237 raft_consensus.cc:2889] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.054837 20208 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:42.100728 14981 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 3 to 7, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 7 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:42.120980 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:42.123546 14981 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 8, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:42.124794 14981 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:42.129276 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.130005 20341 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:42.130478 20341 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.130863 20341 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:42.132700 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.113377 20303 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:42.122784 20066 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:42.140194 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:42.141142 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.099179 20212 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:42.063913 19997 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.146629 15081 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.149080 15158 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:42.152515 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.155159 15157 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:42.014626 20251 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.146584 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.079707 20184 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.118371 20291 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:42.173902 15157 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:04:42.175623 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.183001 15157 pending_rounds.cc:85] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 7
I20260709 13:04:42.176355 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.183339 15157 pending_rounds.cc:107] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:04:42.183390 15084 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.183741 15157 raft_consensus.cc:2889] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.183373 20360 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:42.184338 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.176100 20248 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:04:42.176959 15082 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.118259 20338 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.187908 20338 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.189311 20066 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.147748 20140 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:42.180719 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.127838 20053 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:42.193477 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:42.199090 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 65357.7us
I20260709 13:04:42.182047 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.157280 20353 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.205713 20353 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.205895 20291 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.208168 20353 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:42.157352 20208 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.177088 20045 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:42.211311 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 128959us
I20260709 13:04:42.212247 15079 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:42.205528 20314 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:42.223619 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.225441 15157 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.223786 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:42.225518 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.227694 15082 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.233523 15087 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:04:42.212859 20352 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
I20260709 13:04:42.237083 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:42.237398 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.222497 20140 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:42.248661 15082 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:42.253934 20308 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:42.256340 20308 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:42.200632 20321 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.259358 20321 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.260362 20276 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:42.222555 20183 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.262962 20183 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.263055 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.268493 15843 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:42.268936 15081 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.235782 20153 raft_consensus.cc:2764] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:42.272068 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:42.287616 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 185820us
I20260709 13:04:42.301636 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.304045 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.304821 15084 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.237679 20332 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:04:42.234608 20208 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:42.309155 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61096.9us
I20260709 13:04:42.309787 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.310552 15082 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12.
W20260709 13:04:42.316547 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 262027us
I20260709 13:04:42.317866 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.321799 20332 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:42.322283 20332 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:42.323132 20332 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.324421 20332 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.325662 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.325922 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.336759 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.337299 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.338156 15082 raft_consensus.cc:2368] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:42.341048 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.341274 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.346179 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:42.346325 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.347003 15162 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:42.350381 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.350415 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:42.351377 15162 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.309018 20314 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:42.354184 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:42.465695 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 215624us
W20260709 13:04:42.472712 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:42.473703 20347 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:04:42.473934 20364 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:42.474515 15202 leader_election.cc:400] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:42.475610 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.475783 20184 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:42.477252 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.478025 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.466848 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:42.508082 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 167137us
W20260709 13:04:42.557161 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90398.6us
I20260709 13:04:42.303154 20053 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.345947 20045 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:42.349558 20207 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:42.556329 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 80804.9us
I20260709 13:04:42.572297 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.537505 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.349709 20066 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:42.566563 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.575001 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.575780 15084 raft_consensus.cc:2368] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:42.575891 15086 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.575955 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:42.576912 15082 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.578034 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.540994 20321 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.581864 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:42.582793 15236 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:42.581036 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.558907 20316 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:42.541599 20184 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.562901 20308 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:42.566146 20153 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:42.592180 15086 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:42.595187 15084 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:42.571008 19997 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:42.585669 20324 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.000s
I20260709 13:04:42.604040 20184 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.640030 20183 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.641424 20358 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.285s	user 0.000s	sys 0.000s
I20260709 13:04:42.642575 20343 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.643215 20343 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.489229 20140 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.643946 20045 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:42.653620 20362 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:42.655061 20208 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:42.655723 20338 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.658082 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.659798 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:42.660542 20302 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.002s
I20260709 13:04:42.661101 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:42.664510 15082 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:42.678220 15162 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:42.681576 15051 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.073s (SENT)
I20260709 13:04:42.569124 20364 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:42.682224 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:42.585280 20359 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:42.687801 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.688807 15231 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:42.591871 20066 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.694026 15088 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:42.697234 20371 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:42.701867 20251 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:42.702613 15840 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:42.704348 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.704984 15840 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEARNER]: Advancing to term 12
I20260709 13:04:42.707494 20300 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:42.709113 20338 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.709725 19780 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.710866 19780 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.714391 20246 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:42.715586 20369 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:04:42.640622 20183 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.680265 20302 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:42.591274 20321 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:42.601526 20308 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:42.724370 15840 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEARNER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:42.726774 20368 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.730521 20368 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.733144 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.733880 15842 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.736513 20368 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:42.730538 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.745769 15162 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:42.684063 20353 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.747908 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:42.771438 15236 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:42.737864 15847 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:42.754746 20184 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:42.762501 20317 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.000s
I20260709 13:04:42.763098 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:42.776990 15228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.025s
I20260709 13:04:42.780103 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:42.781549 15202 leader_election.cc:400] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:42.781966 15202 leader_election.cc:403] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:04:42.783481 20382 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.784307 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:42.784255 20382 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.784888 20302 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:42.785089 15236 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.786787 20382 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.789508 15230 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:42.779485 20353 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:42.790184 20341 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.000s
I20260709 13:04:42.790995 15230 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:42.790678 20341 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.791368 15230 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:42.791821 15157 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:42.791903 20353 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.793156 20285 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:04:42.796176 20341 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.791905 20183 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:42.811791 19813 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.814213 15229 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:04:42.780457 15228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.008s	sys 0.025s
I20260709 13:04:42.783564 20343 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.777155 20308 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:42.827545 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:42.827862 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45482.7us
I20260709 13:04:42.819861 15080 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:42.792172 15230 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.822005 15842 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:42.819388 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:42.830861 15082 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.833127 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.835485 15082 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:42.837215 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:42.857686 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.860175 15842 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:42.866791 15230 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.867677 15843 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:42.870185 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:42.857179 20303 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.675s	user 0.004s	sys 0.007s
I20260709 13:04:42.877321 15087 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:42.881321 15236 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:42.856446 20309 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.861577 19813 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.856263 20317 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:42.890084 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:42.890516 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:42.890707 15841 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.892977 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.895726 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.899605 15842 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:42.900864 15843 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:42.903374 15084 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:42.898442 15230 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:42.857543 20342 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:04:42.880620 20183 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:42.908964 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:42.902601 20322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.014s	sys 0.019s
I20260709 13:04:42.895746 19596 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.918869 15083 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:42.920291 15159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1105) took 1902 ms (client timeout 1772 ms). Trace:
I20260709 13:04:42.922010 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1304) took 1043 ms. Trace:
I20260709 13:04:42.930315 15844 rpcz_store.cc:276] 0709 13:04:41.878965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.905146 (+ 26181us) service_pool.cc:224] Handling call
0709 13:04:42.921842 (+1016696us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b540083f254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:42.921950 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213745024}
I20260709 13:04:42.923748 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:42.924429 20381 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:42.925755 20386 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:42.929517 15159 rpcz_store.cc:269] 0709 13:04:41.017713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.025524 (+  7811us) service_pool.cc:224] Handling call
0709 13:04:42.920072 (+1894548us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54000857d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:42.920190 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015772032}
I20260709 13:04:42.933080 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:42.940729 20324 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:42.907315 20317 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:42.889185 20303 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.939850 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:42.945245 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 55280.5us
I20260709 13:04:42.900852 20332 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:04:42.900377 20208 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:42.948064 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:42.929273 20322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.014s	sys 0.019s
I20260709 13:04:42.972043 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:42.972904 15159 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:42.938347 19596 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.936498 20386 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:42.975113 20386 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.975643 19596 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:42.977025 20332 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:42.977548 20332 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:42.978566 20332 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.980798 20332 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.981741 20321 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:42.958770 15233 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.944963 20380 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:42.950541 20300 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:04:42.945078 20183 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:42.985670 20183 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:42.986114 20183 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.958238 20303 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:42.961100 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:42.987370 15154 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:42.938124 20353 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:42.966451 15843 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:42.967598 20381 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:42.967749 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.968024 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.968489 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:42.969208 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:42.898001 20309 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.972177 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:42.955559 20317 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:42.988796 15847 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:42.950681 20324 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:42.994649 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46674us
I20260709 13:04:42.995910 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.007962 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:43.012318 15844 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:43.015625 15843 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:43.015563 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.988327 20302 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 8. Current term is 9.
I20260709 13:04:43.023326 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.024600 15154 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:43.026438 15233 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:43.027868 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.028469 15156 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:42.998778 20212 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:43.029835 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:43.002240 20324 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.038199 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:42.952406 20361 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:43.000501 20353 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:42.952811 20291 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:04:43.038408 15234 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
W20260709 13:04:43.046882 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.046845 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.015233 20184 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.014544 20380 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.051402 15156 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
W20260709 13:04:43.051888 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49238.7us
I20260709 13:04:43.052738 15156 pending_rounds.cc:85] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:43.053078 15156 pending_rounds.cc:107] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:04:43.053347 15156 raft_consensus.cc:2889] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:43.053910 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.056485 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:43.057170 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.042122 20324 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.062250 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:43.065608 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.067050 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:43.065006 20178 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:43.068169 20392 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:04:43.079054 15085 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:43.069993 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:43.050568 20321 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:43.067211 15156 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:43.030817 19780 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:04:43.082152 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.063138 20071 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:43.064428 20246 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:43.074187 20381 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:43.083196 20371 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.089305 20335 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.061160 20251 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:43.084796 19813 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.063612 20276 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:04:43.115262 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 60179.9us
I20260709 13:04:43.063746 20184 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.066963 20380 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.124183 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:43.117055 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53037.5us
I20260709 13:04:43.097105 20071 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:43.100138 20246 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:43.128206 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.126940 20371 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:43.126648 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.131290 19596 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:43.133584 20337 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:43.138589 20390 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:04:43.138535 20184 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:43.139779 20391 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:43.143079 19596 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:43.148186 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.130254 20071 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.129851 20246 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.130582 20371 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:43.148298 20341 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:43.127267 20344 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:04:43.163506 20178 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:43.165891 20178 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.165902 20357 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:43.166821 20382 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:43.177233 20382 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:43.178400 20382 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.179925 20382 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.127931 20353 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.185913 20209 raft_consensus.cc:993] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:04:43.186643 20209 raft_consensus.cc:1081] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:04:43.162603 20309 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.130132 20235 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:43.172250 20380 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.176739 20342 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:43.203478 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79364.7us
I20260709 13:04:43.162576 20371 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.208951 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:43.169798 20341 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:43.204924 20322 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.205276 19997 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:43.217510 15085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1238) took 1210 ms (client timeout 1585 ms). Trace:
I20260709 13:04:43.208035 20342 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:43.189160 20386 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.221544 20383 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:43.189342 20357 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:43.227085 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.11351e+06us
W20260709 13:04:43.226744 15085 rpcz_store.cc:269] 0709 13:04:42.007170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.007294 (+   124us) service_pool.cc:224] Handling call
0709 13:04:43.217336 (+1210042us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b540034a154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.217431 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127302016}
I20260709 13:04:43.226707 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:43.237465 15158 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.239476 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:43.241135 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 119756us
I20260709 13:04:43.242389 15158 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:43.242532 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:43.228232 20322 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.244809 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.241914 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.245574 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.247377 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.246021 15154 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:43.252458 20342 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.253554 20342 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.253647 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.233343 20291 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.230417 20383 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:43.230261 19997 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.268863 20383 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.270427 20383 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:43.271029 19997 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.271591 20402 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:43.265161 20322 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.260859 20357 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.267220 15848 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:43.257872 20251 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:43.267714 20386 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.268589 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1070) took 2369 ms. Trace:
I20260709 13:04:43.238253 20381 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:43.248955 20289 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:43.276717 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.276798 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:43.293005 15843 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:43.288115 15237 rpcz_store.cc:276] 0709 13:04:40.899291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.912468 (+ 13177us) service_pool.cc:224] Handling call
0709 13:04:40.913808 (+  1340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.121818 (+1208010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.268324 (+1146506us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002d4954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:43.268505 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2752733440}
W20260709 13:04:43.297655 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 59144.8us
I20260709 13:04:43.264099 20364 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:43.302806 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:43.302994 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.306365 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.298448 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.315084 20371 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.317929 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:43.318748 15844 raft_consensus.cc:2368] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12.
I20260709 13:04:43.319443 20324 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.263525 20341 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:43.321455 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.322350 15201 leader_election.cc:400] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12.
I20260709 13:04:43.322670 15201 leader_election.cc:403] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:43.303990 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:43.329702 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.287484 20251 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.292177 20289 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.339339 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:43.340538 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.337801 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:43.386759 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47508.1us
I20260709 13:04:43.395936 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.309535 20364 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:43.388348 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 50638.9us
I20260709 13:04:43.411291 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.344780 20188 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.420709 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:43.429410 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.345451 19997 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.337688 20251 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.385169 20334 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.387228 20344 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.430255 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:43.432588 15158 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:43.332594 20360 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12.
W20260709 13:04:43.438925 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 140520us
I20260709 13:04:43.336395 20357 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.390611 20338 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:43.397691 20403 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:43.433410 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:43.447561 15155 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.439823 15813 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12.
I20260709 13:04:43.448042 15813 leader_election.cc:403] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:43.438510 15157 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.332108 20178 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:04:43.386520 20291 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:04:43.443037 15082 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.442883 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1245) took 1395 ms. Trace:
I20260709 13:04:43.460876 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.430984 20399 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:43.438108 19997 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:43.442555 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48327.8us
I20260709 13:04:43.467167 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.468277 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.401328 20322 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:43.461158 15079 rpcz_store.cc:276] 0709 13:04:42.051164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.061201 (+ 10037us) service_pool.cc:224] Handling call
0709 13:04:43.442691 (+1381490us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540011add4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.442825 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332194176}
I20260709 13:04:43.463922 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:43.470925 15162 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.398187 20071 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.445530 20308 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.448470 20217 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:43.458081 20412 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:04:43.401152 20386 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:04:43.461311 20140 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:43.486793 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47110.8us
I20260709 13:04:43.472761 19997 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.476126 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.446522 20344 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.458413 19596 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.475467 20212 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12.
I20260709 13:04:43.461788 20338 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:43.521067 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:43.401584 20289 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.466054 20371 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.451125 20025 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.538864 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:43.539206 20400 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.539739 15084 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.540253 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:43.505767 15162 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:43.521332 20140 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:43.542088 20140 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.431360 20183 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.513314 20401 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:43.516453 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:43.601640 15234 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:43.517319 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:43.502419 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.451802 20381 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.518101 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:43.482084 20380 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:43.523581 20416 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:43.523932 20373 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.524080 15848 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:04:43.524859 20324 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:43.529492 20413 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:04:43.466539 20364 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.618956 20364 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:43.499737 19780 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.620445 19780 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.536355 15235 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:43.501181 15845 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:43.622805 19780 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.623925 20401 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.539851 20400 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.538938 20025 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.546075 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.547003 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.631402 20212 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.631943 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:43.632186 20025 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:43.606987 20360 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.632720 15079 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.633349 20360 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.551748 20217 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:43.635535 20217 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.635689 20360 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.553201 20368 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.638612 20317 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:43.558996 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1258) took 1233 ms. Trace:
I20260709 13:04:43.638788 20368 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.639464 15088 rpcz_store.cc:276] 0709 13:04:42.325851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.328922 (+  3071us) service_pool.cc:224] Handling call
0709 13:04:43.558804 (+1229882us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540011b2d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.558936 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274478976}
I20260709 13:04:43.640614 20380 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:43.641333 20380 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:43.641371 20053 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:43.558483 20338 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.524619 20408 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.561180 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2783) took 1088 ms. Trace:
W20260709 13:04:43.562331 20398 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.390s	user 0.002s	sys 0.002s
I20260709 13:04:43.644565 15086 rpcz_store.cc:276] 0709 13:04:42.473222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.596410 (+123188us) service_pool.cc:224] Handling call
0709 13:04:43.558942 (+962532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.559126 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.559162 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.559219 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:43.559263 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.559337 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.560962 (+  1625us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b540011b2d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.561106 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212684544}
W20260709 13:04:43.562696 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2344) took 1521 ms (client timeout 1917 ms). Trace:
W20260709 13:04:43.646291 15233 rpcz_store.cc:269] 0709 13:04:42.041028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.067819 (+ 26791us) service_pool.cc:224] Handling call
0709 13:04:43.562514 (+1494695us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400295c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.562628 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047040768}
W20260709 13:04:43.564520 15080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1242) took 1556 ms (client timeout 1515 ms). Trace:
I20260709 13:04:43.567211 15232 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:43.572083 20289 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.573722 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.574506 20071 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:43.576968 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.470173 20334 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.578801 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.582715 20344 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:43.584204 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.584996 20418 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:04:43.637989 19780 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:43.593407 20415 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 13:04:43.518978 19596 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.517534 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:43.614295 20407 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 13:04:43.500859 20308 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.614431 20246 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.616743 20183 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.609830 15084 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:04:43.628625 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 126318us
W20260709 13:04:43.638077 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 92108.9us
W20260709 13:04:43.654755 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 206305us
I20260709 13:04:43.655005 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:43.720285 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.720638 15848 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:04:43.647739 15080 rpcz_store.cc:269] 0709 13:04:42.008345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.009227 (+   882us) service_pool.cc:224] Handling call
0709 13:04:43.564304 (+1555077us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540011a3d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:43.564439 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533868288}
W20260709 13:04:43.687141 20382 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.507s	user 0.013s	sys 0.000s
I20260709 13:04:43.726166 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:43.676071 20368 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.662573 15082 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:43.730757 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.667739 15079 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:43.591416 20066 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:43.675329 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 96640.5us
W20260709 13:04:43.678187 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 217419us
I20260709 13:04:43.733469 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:43.693348 15203 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.768s (SENT)
I20260709 13:04:43.734484 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:43.712785 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:43.717303 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.657029 15160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34534
I20260709 13:04:43.726675 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:43.741802 15235 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:43.661247 15157 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:43.727910 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:43.688076 20184 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.686128 20361 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:04:43.749928 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.750384 20361 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:43.751354 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.753018 20291 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:43.753756 20291 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:43.753790 20399 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:43.754467 20291 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:43.754875 20291 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:43.739560 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:43.755264 20291 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.757201 20399 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:43.757719 20399 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:43.757869 20209 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:43.752523 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.624615 20401 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.746743 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:43.760717 15081 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:43.643139 20373 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.763449 15080 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:43.715605 19596 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:43.767005 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.772070 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:43.772704 15080 raft_consensus.cc:2376] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14. Re-sending same reply.
I20260709 13:04:43.779321 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:43.748508 20335 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:43.781735 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.782100 15079 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.761843 20276 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.785212 15156 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:43.788164 15839 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:43.717007 19780 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.793977 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.767580 20361 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:43.795248 15084 raft_consensus.cc:2410] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:43.797853 15080 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:43.800261 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:43.801728 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.801401 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:43.802889 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:43.802830 15084 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:43.803642 15079 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:43.783902 20335 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:43.798405 20361 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.798887 20045 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.758687 20399 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.805701 20383 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:43.812044 20353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.003s	sys 0.021s
I20260709 13:04:43.808553 20066 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:43.808069 15080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:56652
W20260709 13:04:43.821410 20321 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.008s	sys 0.025s
I20260709 13:04:43.794451 20276 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.814966 20335 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.817898 20403 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:43.829138 20341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.018s
I20260709 13:04:43.933347 19780 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.936214 20373 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:43.937255 20308 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:43.938413 20246 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.780615 20380 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:43.823169 20334 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:43.823554 20361 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.942641 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:43.902063 20353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.004s	sys 0.022s
W20260709 13:04:43.940469 20321 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.009s	sys 0.025s
I20260709 13:04:43.953225 15157 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:44.032799 20400 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.167958 15157 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:44.177627 20419 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
W20260709 13:04:44.183696 20217 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:04:44.184809 20217 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.186028 20291 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.188793 20291 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:44.208873 15160 raft_consensus.cc:606] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:44.208976 20401 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.209450 15160 raft_consensus.cc:614] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:44.210117 20422 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:04:44.212352 20217 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.835423 20383 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:43.952502 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 214404us
W20260709 13:04:44.230937 20443 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:43.953722 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:43.945638 20341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.005s	sys 0.018s
W20260709 13:04:44.099252 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 405931us
W20260709 13:04:44.260931 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.962568 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.263126 15079 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:44.101622 20025 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.102084 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.105857 19997 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.587s	user 0.017s	sys 0.002s
W20260709 13:04:44.106254 20251 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.658s	user 0.006s	sys 0.014s
I20260709 13:04:44.108031 20314 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:44.109360 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 362702us
I20260709 13:04:44.110045 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:44.111245 20188 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.010s
I20260709 13:04:44.258831 20383 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:44.276489 20188 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:44.114692 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:43.961668 20321 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:44.119043 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 375514us
I20260709 13:04:44.122084 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:44.281508 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 159491us
I20260709 13:04:44.281667 20321 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:44.282747 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.282881 20309 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.283880 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.283568 20309 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.284245 20437 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:44.285084 20276 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.286072 20309 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.122768 20071 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:43.896188 20045 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.296227 20045 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:43.947489 20066 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:44.297479 20066 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:43.985401 20289 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:44.137497 15080 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:44.147312 20403 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.298878 15080 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:43.953562 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 196178us
I20260709 13:04:44.298887 19997 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:44.250937 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.303391 19997 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.303402 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.272624 15086 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:44.272794 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:44.241716 20136 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.305925 20136 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.308214 20136 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.275427 20251 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.310374 15085 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:44.310642 15080 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.311975 20251 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:44.267486 19596 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.315291 19596 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:44.319079 20436 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.319648 20436 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:44.320097 20424 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:04:44.324432 20314 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:44.376583 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 103874us
I20260709 13:04:44.381870 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:44.381897 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 107180us
W20260709 13:04:44.383675 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 110006us
I20260709 13:04:44.392900 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:44.393850 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.397563 15082 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:44.399339 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.405628 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.407465 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.408293 15087 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:44.416395 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.284762 20437 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:44.420783 15079 raft_consensus.cc:2410] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:44.421602 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:44.422842 15082 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:44.423924 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.428444 15840 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:44.384783 20382 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.433646 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:44.435420 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:44.438621 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:44.388399 20344 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:44.292606 20335 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.446921 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.393076 20309 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.274889 20025 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.480114 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.471598 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1313) took 1469 ms. Trace:
I20260709 13:04:44.479696 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:44.493037 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.493723 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.493884 15846 raft_consensus.cc:2368] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15.
I20260709 13:04:44.494787 15840 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:44.476356 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.490049 15082 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:44.493745 15842 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:44.478946 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:44.410072 20432 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.492303 15839 rpcz_store.cc:276] 0709 13:04:43.002403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.002604 (+   201us) service_pool.cc:224] Handling call
0709 13:04:43.024395 (+ 21791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.791668 (+767273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.471120 (+679452us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:44.471509 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755595520}
I20260709 13:04:44.426817 20276 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.430645 20246 raft_consensus.cc:2764] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:44.431443 20437 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:44.476938 20217 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:44.515231 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.443734 20341 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:44.516732 15082 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:44.406769 20322 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:04:44.518707 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.527297 15843 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.477087 20402 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:44.458542 20364 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.001s
I20260709 13:04:44.426537 20251 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:04:44.531905 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.440078 19997 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:04:44.505878 20338 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:44.589833 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 71226.6us
I20260709 13:04:44.591728 15161 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:44.592695 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.447945 20382 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.505650 20317 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:44.597445 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.601378 15161 raft_consensus.cc:2410] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:44.598063 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.598150 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.529834 20344 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:44.533849 20217 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:44.610090 15086 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.602720 15156 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:44.508831 20309 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.539067 20341 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:44.594599 20408 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.588474 20380 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.587065 20136 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.537575 20322 consensus_queue.cc:1243] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:44.616432 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.614643 15843 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:44.599526 20402 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:44.621570 15159 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:44.507977 20314 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.520676 20432 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:44.617297 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 355007us
W20260709 13:04:44.616250 20361 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.000s
I20260709 13:04:44.602243 20360 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.522734 20276 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:04:44.615362 20217 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:44.639176 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:44.622327 20341 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:44.659471 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:44.648808 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 251810us
I20260709 13:04:44.634358 20401 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:44.660740 15052 leader_election.cc:400] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15.
I20260709 13:04:44.664178 15052 leader_election.cc:403] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:44.637066 20184 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.662050 15083 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:44.630172 20436 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:44.652281 20053 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.684101 20317 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:44.701270 20402 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.703884 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2799) took 1394 ms. Trace:
I20260709 13:04:44.707327 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.634867 20380 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.704517 15085 rpcz_store.cc:276] 0709 13:04:43.309284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.309437 (+   153us) service_pool.cc:224] Handling call
0709 13:04:43.309747 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.310803 (+1001056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.703542 (+392739us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001905d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:44.703796 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300657536}
I20260709 13:04:44.663924 20401 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:44.646148 20136 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.733250 15161 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:44.674216 20314 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:44.670771 20400 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:44.773063 20400 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.733533 20383 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.773612 20400 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:44.775136 20383 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:44.684103 20309 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.667685 20344 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.711198 20053 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.669493 20321 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:44.779105 20321 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.779584 20321 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:44.779997 20053 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.782617 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:44.783475 15162 raft_consensus.cc:2368] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14.
I20260709 13:04:44.784251 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:44.654402 20361 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.750801 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.738574 20335 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:44.792368 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.793344 15814 leader_election.cc:400] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14.
I20260709 13:04:44.793687 15814 leader_election.cc:403] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:04:44.729856 20444 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:44.794315 20444 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.797220 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.712347 20317 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.768950 20459 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.798905 20276 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:44.769384 20398 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:04:44.799757 20317 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:44.769577 20382 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.770197 20025 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:44.661636 20360 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:44.800679 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2798) took 1610 ms. Trace:
I20260709 13:04:44.772269 20432 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.758368 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.812916 20136 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.813665 20045 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14.
I20260709 13:04:44.814465 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:44.815279 20025 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:44.815691 20025 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.816052 20025 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:44.788255 15162 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:44.788949 15158 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:04:44.821911 15158 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:44.796598 20444 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.801101 15083 rpcz_store.cc:276] 0709 13:04:43.190223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.190520 (+   297us) service_pool.cc:224] Handling call
0709 13:04:43.212557 (+ 22037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.666772 (+1454215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.800467 (+133695us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400191754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:44.800613 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3324132224}
I20260709 13:04:44.822702 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:44.823249 15158 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:44.823362 15233 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:44.798620 20459 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:44.769137 15087 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:44.810323 20360 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.836247 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:44.829015 15839 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:44.829591 15842 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:44.829044 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.755752 20401 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.850903 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.835548 15848 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:44.821545 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 160932us
I20260709 13:04:44.850006 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.850155 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.854538 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:44.856109 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:44.857348 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.853376 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.852152 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.859042 15842 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:44.859992 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.854424 15846 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:04:44.836452 20451 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 13:04:44.856582 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:44.872975 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
W20260709 13:04:44.836902 20449 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
I20260709 13:04:44.883539 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:44.885529 15233 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:44.891680 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.883792 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.903571 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:44.894212 15203 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.969s (SENT)
I20260709 13:04:44.898546 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:44.910280 15233 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:04:44.910223 15848 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:44.910806 15233 pending_rounds.cc:85] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 10
I20260709 13:04:44.911173 15233 pending_rounds.cc:107] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:04:44.903741 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.911487 15233 raft_consensus.cc:2889] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:44.911773 15079 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:04:44.914124 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.845378 20184 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15.
W20260709 13:04:44.916399 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 51440.4us
I20260709 13:04:44.928740 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.918205 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:44.933041 15233 raft_consensus.cc:2410] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:44.920853 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:44.920974 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:44.943248 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.951314 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:44.954819 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:44.823518 20383 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:04:44.879946 19596 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:44.880066 20432 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.955704 15237 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:44.897583 20317 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.969770 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:44.975157 15163 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:44.897540 20408 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:44.901453 20045 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.897585 20364 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:44.976284 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.980657 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2796) took 1874 ms. Trace:
I20260709 13:04:45.017038 15081 rpcz_store.cc:276] 0709 13:04:43.105787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.105945 (+   158us) service_pool.cc:224] Handling call
0709 13:04:43.106301 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.765909 (+659608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.980406 (+1214497us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:44.980587 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504335232}
I20260709 13:04:44.897215 20360 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:04:44.917541 20053 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:44.920596 20335 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:04:45.031152 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:44.981846 20436 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:45.038753 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.047015 15081 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:45.051318 15086 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:04:45.054682 20459 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:45.056067 20361 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:44.980198 20317 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.065717 20025 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.067070 15841 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:45.068570 20025 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.068697 20360 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:45.064786 20462 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:04:45.067155 20317 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.024979 20467 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.078341 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.034165 20071 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:45.083096 15081 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:44.976446 20432 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:45.107867 20432 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:45.109946 20458 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:45.112386 20380 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.115013 15088 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:45.096148 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.051287 20436 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.118584 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:45.124080 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.140183 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.143397 15088 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
W20260709 13:04:45.143478 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 259812us
I20260709 13:04:45.029055 20364 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.085683 20071 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.144263 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:45.080298 20466 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:45.049981 20324 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:04:45.123241 20183 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.150892 20437 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 13:04:45.140292 20436 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:45.227093 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:45.117218 20465 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:04:45.240090 20460 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:45.240942 20460 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.242832 20360 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:45.243343 20360 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:45.244270 20360 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:45.246235 20360 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.247565 20379 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:45.256532 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34534 (request call id 0) took 1631 ms. Trace:
I20260709 13:04:45.257175 15160 rpcz_store.cc:276] 0709 13:04:43.624658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.626750 (+  2092us) service_pool.cc:224] Handling call
0709 13:04:45.256216 (+1629466us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54003e0654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9de853 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:45.256387 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075264384}
I20260709 13:04:45.180184 20071 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:45.265954 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 409451us
I20260709 13:04:45.267757 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:45.196592 20344 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.196897 20335 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:45.220857 20432 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:45.289876 20432 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.092217 20467 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.293502 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.275444 20466 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:45.277148 20364 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.279664 20324 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:45.295055 20467 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:45.295888 20314 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.000s
I20260709 13:04:45.298218 20344 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.229539 20183 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.282286 20342 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:45.237505 20437 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.306879 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.308753 15082 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:45.306449 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:45.292634 20432 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.265357 20460 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:45.315704 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 410377us
I20260709 13:04:45.321995 20251 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.322686 20423 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:45.322602 20251 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.323262 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.324048 15085 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:45.325536 20251 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.323436 20423 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.326800 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:45.327343 15082 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:45.328595 20423 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.334136 15082 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:45.326161 20251 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.338732 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:45.339318 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 445111us
I20260709 13:04:45.337795 15840 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:45.306769 20342 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:45.339949 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:45.306402 20400 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.002s
I20260709 13:04:45.310946 20344 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.312345 20466 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:45.341464 20321 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.001s
I20260709 13:04:45.349529 20276 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.315014 20401 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.343694 20379 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:45.357198 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 428508us
I20260709 13:04:45.357240 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:45.355183 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.368965 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:45.369179 15233 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:45.370028 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.369220 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.370939 15845 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:45.371086 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.371801 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.372018 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.372215 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.372958 15846 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:45.373337 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.372385 15236 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:45.386439 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:45.388410 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:45.391294 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:45.374343 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.393106 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.395766 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:45.396133 15236 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:45.398501 15237 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:45.358104 20025 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:45.344241 20251 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:45.387765 20341 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
I20260709 13:04:45.401854 15230 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:45.355746 20460 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.373826 20314 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:45.357928 20400 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.404883 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:45.411782 20342 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:45.412801 20474 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:45.413223 20342 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.413298 20400 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:45.413980 20361 consensus_queue.cc:1175] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 13 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:45.416791 20459 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:45.417186 15081 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:45.417249 20459 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.417802 20459 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.411154 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:45.422379 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 495406us
I20260709 13:04:45.423270 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:45.401110 20467 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:45.438041 15162 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:45.402341 20464 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:45.439893 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:45.356914 20477 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:04:45.445201 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:45.390419 20432 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:04:45.359205 19596 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:45.365406 20321 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.478929 20432 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:45.479355 20432 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:45.391053 20276 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.451282 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:45.459357 20437 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:45.460390 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:45.461107 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.462030 15230 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:45.446976 20361 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:45.462738 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.462944 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:45.463680 20140 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:45.462441 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:45.494776 15161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.003s	sys 0.028s
I20260709 13:04:45.410969 20341 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:45.357676 20466 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:45.438755 20467 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:45.424867 20460 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:45.424995 20217 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.003s
I20260709 13:04:45.438458 20364 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:45.520398 15083 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:45.484747 15160 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.492439 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:45.462568 20071 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.447636 20464 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:45.495150 15161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.003s	sys 0.028s
I20260709 13:04:45.496119 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:45.494367 15231 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:45.494897 15229 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:45.473742 20251 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.523011 15080 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.535346 20328 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:45.537278 15844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de505aa1e895447486f53629c5269998"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:45.540683 15082 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:45.546541 15844 raft_consensus.cc:606] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:45.541797 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:45.480372 20432 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.546916 15844 raft_consensus.cc:614] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:45.494105 20361 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.581660 15162 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:45.582543 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:45.582217 15156 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:45.517544 20467 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:45.585248 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56652 (request call id 1267) took 1747 ms. Trace:
I20260709 13:04:45.520689 20364 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:45.586061 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90016.8us
I20260709 13:04:45.624811 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:45.588464 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:45.590021 15080 rpcz_store.cc:276] 0709 13:04:43.799063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.799429 (+   366us) service_pool.cc:224] Handling call
0709 13:04:44.960450 (+1161021us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54001b12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9dba3d 00007f0d449abcd0 00007f0d3df30254 00007f0d3df301e2 00007f0d3df30111 00007f0d3df3008c 00007f0d3df2f342 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:45.522836 (+562386us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4c0023a2c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3dcb0a20 00007f0d3dcc1a29 00007f0d3dcc213f 00007f0d3dc862ab 00007f0d3dcb3f3c 00007f0d3dcb43fe 00007f0d3df4188b 00007f0d3f940495 00007f0d3f99130f 00007f0d3f9dba55 00007f0d449abcd0
0709 13:04:45.540777 (+ 17941us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530252032}
W20260709 13:04:45.585644 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48699.3us
I20260709 13:04:45.545627 20360 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:45.594249 20341 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.594419 20480 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.596055 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:04:45.635826 15082 raft_consensus.cc:606] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:45.596674 20342 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:45.597172 20466 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.598899 20276 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.600051 20436 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.614703 20321 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.518201 20140 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.615551 19596 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.586400 20361 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:45.618930 20460 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:45.628451 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:45.581764 20400 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:45.582165 20464 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:45.636011 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.705543 15161 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:45.630038 20251 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:04:45.706183 15847 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:45.583160 20324 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:45.501286 20459 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.537632 20217 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:45.622648 20053 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:45.628274 20364 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.726163 15163 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:45.622895 20317 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:04:45.716391 20460 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:04:45.641528 20480 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.702039 20309 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:45.655537 20432 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:45.709911 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:45.639349 20479 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:04:45.711539 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:45.743960 15841 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:45.652462 15160 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:45.744333 15841 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:45.709544 20400 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.729074 20276 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.757370 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 310157us
I20260709 13:04:45.722947 20053 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:45.730345 20285 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:45.766479 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.769896 15843 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:45.723457 20322 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:45.776221 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.794593 15844 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:45.776624 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 151893us
I20260709 13:04:45.722402 20324 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.777689 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:45.802450 15842 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.768069 20053 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.780073 15841 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:45.769400 20285 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:45.838850 15841 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:45.774513 20456 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.006s	sys 0.010s
I20260709 13:04:45.817194 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:45.824234 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:45.826202 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:45.842031 15843 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:45.832342 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:45.843127 15839 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.844702 15844 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:45.855377 20456 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.011s
I20260709 13:04:45.772204 20444 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:04:45.848807 15842 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:45.859809 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:45.811969 20217 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:45.865607 15087 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:45.773947 20276 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:45.872280 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 240175us
I20260709 13:04:45.837335 20071 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:45.873983 15081 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:45.874879 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:45.848564 20309 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:45.882478 15839 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:45.889163 20309 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:45.987947 20460 consensus_queue.cc:1243] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:04:46.007107 20496 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 13:04:46.008176 20489 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:04:46.009618 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2370) took 1119 ms. Trace:
W20260709 13:04:46.009897 20486 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
I20260709 13:04:46.010329 15235 rpcz_store.cc:276] 0709 13:04:44.889819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.890064 (+   245us) service_pool.cc:224] Handling call
0709 13:04:44.903424 (+ 13360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.905940 (+  2516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.905974 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.906429 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.426917 (+520488us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:45.426972 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.427066 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.009320 (+582254us) raft_consensus.cc:1685] finished
0709 13:04:46.009352 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.009552 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6780,"raft.queue_time_us":2226,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":58481,"thread_start_us":462588,"threads_started":1,"wal-append.queue_time_us":680723}
W20260709 13:04:46.028424 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 254846us
W20260709 13:04:45.989277 20455 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:04:46.054123 20505 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:46.055630 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1273) took 1655 ms. Trace:
W20260709 13:04:46.056485 20494 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:04:46.056056 15084 rpcz_store.cc:276] 0709 13:04:44.400701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.400969 (+   268us) service_pool.cc:224] Handling call
0709 13:04:44.401404 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.424897 (+ 23493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.424950 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.425597 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.418917 (+993320us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b18003b3c80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:45.755419 (+336502us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:45.814068 (+ 58649us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.853549 (+ 39481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.853630 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.055362 (+201732us) raft_consensus.cc:1685] finished
0709 13:04:46.055455 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.055577 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67553,"prepare.queue_time_us":88357,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":153458,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":22421,"spinlock_wait_cycles":55102080,"thread_start_us":336247,"threads_started":1,"wal-append.queue_time_us":1602802}
I20260709 13:04:46.058219 20324 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:46.059110 20500 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:04:46.060341 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2807) took 1146 ms. Trace:
I20260709 13:04:46.060705 15079 rpcz_store.cc:276] 0709 13:04:44.914149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.958733 (+ 44584us) service_pool.cc:224] Handling call
0709 13:04:44.959207 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.312828 (+353621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.317804 (+  4976us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.320817 (+  3013us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.633457 (+312640us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:45.633508 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.633597 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.060117 (+426520us) raft_consensus.cc:1685] finished
0709 13:04:46.060137 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.060282 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13817,"raft.queue_time_us":125918,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":17279,"thread_start_us":574295,"threads_started":2,"wal-append.queue_time_us":416501}
I20260709 13:04:46.037406 20436 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.062307 20444 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:46.034615 20459 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.064690 20480 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.044783 15844 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:04:46.083668 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52802.3us
I20260709 13:04:46.089538 20401 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.085582 20400 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.052260 20140 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.098599 20183 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.101517 20473 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:46.102000 20473 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.102944 20473 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.104059 20473 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.108717 20400 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.109369 20400 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.111423 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:46.108006 20361 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.118461 20400 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.121930 15160 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:46.009071 20309 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.118512 20276 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.129733 20136 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:46.114919 20183 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.138430 20361 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.150020 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:46.151039 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:46.159265 15081 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:46.160502 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.164204 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.164932 20309 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.165586 15086 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:46.166872 20276 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:46.180598 15160 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:46.184737 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:46.186978 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:46.195393 20379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.020s	sys 0.010s
W20260709 13:04:46.195986 20379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.020s	sys 0.010s
I20260709 13:04:46.197947 20379 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:46.199213 20379 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.222327 15160 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:46.174440 20467 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.009s
W20260709 13:04:46.217764 20516 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:04:46.180047 20400 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.143106 20183 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.217284 20456 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:46.153573 20421 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:04:46.226065 20136 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:46.252015 20136 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.221848 20324 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:04:46.249902 15841 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:46.218664 20235 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:04:46.267048 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 392201us
I20260709 13:04:46.267846 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:46.227994 20467 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.275751 20467 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.276346 20467 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:46.224010 20408 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:04:46.258792 20456 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.279042 20456 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.240126 20400 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.281109 20183 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.226236 20276 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:04:46.226133 19780 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:04:46.292415 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 118261us
W20260709 13:04:46.300602 15126 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.693s (SENT)
I20260709 13:04:46.301340 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.301355 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:46.302222 15159 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:46.304438 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:46.269423 20518 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:46.328753 15126 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.776s (SENT)
I20260709 13:04:46.331991 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:46.261595 20342 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:46.335018 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 67222.1us
I20260709 13:04:46.336512 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:46.301496 20467 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:46.303949 20464 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.014s
I20260709 13:04:46.315564 20525 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.303015 20509 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.313917 20308 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.328577 20314 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.350216 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.335985 20140 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.333216 20353 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:46.352046 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 534963us
I20260709 13:04:46.362349 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2383) took 1568 ms. Trace:
I20260709 13:04:46.343755 20361 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:46.335242 20520 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.340240 20467 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:46.413043 20432 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.659s	user 0.003s	sys 0.018s
I20260709 13:04:46.413415 20467 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:46.384728 20353 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.382952 20025 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.381479 20314 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.383137 15156 rpcz_store.cc:276] 0709 13:04:44.794071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.812833 (+ 18762us) service_pool.cc:224] Handling call
0709 13:04:44.813181 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.590035 (+776854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.361989 (+771954us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540033e854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:46.362266 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768200064}
I20260709 13:04:46.427554 15845 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:46.386773 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.389362 20335 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.381750 20518 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.345247 20464 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:46.393271 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 560017us
I20260709 13:04:46.395089 20188 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.396598 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.397653 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.381055 20526 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:46.399091 20071 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:46.400213 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:46.492457 15843 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:46.401253 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.402211 15842 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:46.405047 20423 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.406068 19596 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.406683 20378 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.337318 20342 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:46.416968 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.339275 20360 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:04:46.357499 20045 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.432144 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1333) took 1063 ms. Trace:
I20260709 13:04:46.424100 20353 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.359259 20308 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:46.446251 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59577.8us
W20260709 13:04:46.447054 15840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1320) took 1945 ms (client timeout 1909 ms). Trace:
I20260709 13:04:46.398526 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:46.534152 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:46.448969 20335 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.465008 20188 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.482578 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:46.486017 20314 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.489297 20459 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.547761 20459 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.548362 20317 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:46.548967 20317 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:46.550099 20317 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.551359 20317 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.551604 20308 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.530500 20235 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.421298 20432 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.554497 20188 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.543741 20540 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:04:46.554916 20432 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.557920 20432 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.419692 20324 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.552682 20235 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.559177 20324 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.493358 15839 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:46.498761 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.561153 15843 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:46.561542 20324 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.561307 20235 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.512020 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:46.516108 15163 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:46.525771 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:46.566605 15086 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.526520 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:46.569254 15079 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:46.528911 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:46.506273 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:46.518692 19997 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:46.534555 15840 rpcz_store.cc:269] 0709 13:04:44.480729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.508693 (+ 27964us) service_pool.cc:224] Handling call
0709 13:04:46.358002 (+1849309us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54006f0254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:46.426195 (+ 68193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764094720}
W20260709 13:04:46.573356 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44575.2us
I20260709 13:04:46.530471 15848 rpcz_store.cc:276] 0709 13:04:45.369179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.369294 (+   115us) service_pool.cc:224] Handling call
0709 13:04:45.385496 (+ 16202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.385715 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:45.385976 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.351966 (+965990us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800045040. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:46.431646 (+ 79680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:46.431722 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:46.431789 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.431899 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.432058 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11841,"prepare.queue_time_us":10891,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":5024,"raft.queue_time_us":1003038,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"spinlock_wait_cycles":2808704,"thread_start_us":79379,"threads_started":1}
I20260709 13:04:46.538147 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.424726 20520 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.447995 20251 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:04:46.588091 15086 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:46.589500 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.547039 20541 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.382472 20509 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.562081 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.599325 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.600817 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:46.562492 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.594784 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:46.602464 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.603216 15839 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:46.604245 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.537158 20045 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.570232 15842 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.607344 15079 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:46.608127 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:46.608610 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.534766 15844 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.481000 20518 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.576659 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.611435 15085 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:46.577484 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.577632 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.575762 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:46.579418 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:46.580075 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.523633 20526 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.614694 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.521386 20071 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.521219 20423 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.644950 15237 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.580951 20437 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:46.615204 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:46.566605 20335 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:46.624786 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44797.3us
I20260709 13:04:46.651995 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.653465 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:46.617587 20053 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.012s
W20260709 13:04:46.626066 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46869.5us
I20260709 13:04:46.661394 20335 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:46.661900 20335 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.662334 20335 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.664867 20481 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.665462 20481 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.640197 20209 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.666491 20209 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.636396 20188 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:46.669351 20188 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:46.670583 20526 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.670495 20188 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.573243 19997 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.671739 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.617296 20383 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:46.593657 20314 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:46.647286 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.597970 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.614465 20140 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:46.679728 20361 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.680763 20383 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:46.680853 20071 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.681761 20383 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.683511 20045 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.684007 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.598419 20436 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:04:46.685039 20436 consensus_queue.cc:1243] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:46.685091 20353 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.526244 19596 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.597518 20324 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:04:46.687644 19596 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.586725 20309 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.631058 15844 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:46.624164 20360 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:46.695158 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.667668 20481 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:46.638891 20364 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.904s	user 0.022s	sys 0.000s
I20260709 13:04:46.676990 20053 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:46.685650 20473 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.002s
I20260709 13:04:46.675976 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.660434 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:46.705235 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45107.8us
I20260709 13:04:46.705999 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:46.689603 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.596572 15841 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:46.623427 15842 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:46.690528 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.691656 15840 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:46.694376 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.694554 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.698606 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:46.699965 20209 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.636974 20432 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.530748 20378 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.644071 19780 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:46.673434 20464 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.635331 20467 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.613667 20235 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:04:46.708801 15848 raft_consensus.cc:2393] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:46.711071 15079 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:46.711913 15085 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:46.710781 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:46.715210 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:46.715353 15203 connection.cc:256] RPC call timeout handler was delayed by -1.0503880539999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:46.725200 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.687649 20353 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.695477 20360 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.695187 20309 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.729291 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.729413 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.730049 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.731107 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.704137 20328 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:46.727592 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:46.731305 15235 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:46.703759 20518 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.771559 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.772285 15235 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:46.724428 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:46.731961 15233 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:46.641243 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:46.716507 20429 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:46.733697 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.801678 15230 raft_consensus.cc:2376] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12. Re-sending same reply.
I20260709 13:04:46.728675 20360 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.734223 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.735005 20276 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:46.736791 20473 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:46.737352 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.738206 20184 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:46.739583 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.740352 20322 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:46.744191 20483 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.977s	user 0.004s	sys 0.000s
I20260709 13:04:46.752149 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.754846 20309 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.726846 20432 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.756434 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1128) took 1308 ms. Trace:
I20260709 13:04:46.757030 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2125) took 1358 ms. Trace:
I20260709 13:04:46.752216 20314 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:46.714608 20324 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:46.775964 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51617.1us
I20260709 13:04:46.860087 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:46.777001 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47681.3us
I20260709 13:04:46.842505 15236 rpcz_store.cc:276] 0709 13:04:45.398665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.398790 (+   125us) service_pool.cc:224] Handling call
0709 13:04:45.399269 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.402375 (+  3106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.402413 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.409504 (+  7091us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.763653 (+354149us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:45.814670 (+ 51017us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.252451 (+437781us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b180041e9c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:46.331740 (+ 79289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.331831 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.753992 (+422161us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b180041e9c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:46.756824 (+  2832us) raft_consensus.cc:1685] finished
0709 13:04:46.756863 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.756979 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37825,"prepare.queue_time_us":74187,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":4019,"raft.queue_time_us":453728,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":3464,"spinlock_wait_cycles":19639424,"thread_start_us":388330,"threads_started":2,"wal-append.queue_time_us":419364}
I20260709 13:04:46.862232 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.841065 15228 rpcz_store.cc:276] 0709 13:04:45.448337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.448486 (+   149us) service_pool.cc:224] Handling call
0709 13:04:45.448813 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.588344 (+139531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.588372 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.588809 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.806704 (+217895us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:45.806744 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.756058 (+949314us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400175a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:46.756106 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.756149 (+    43us) raft_consensus.cc:1685] finished
0709 13:04:46.756162 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.756363 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4425,"raft.queue_time_us":10319,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":309704576,"thread_start_us":150611,"threads_started":1,"wal-append.queue_time_us":413978}
I20260709 13:04:46.721637 20532 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:04:46.777899 20353 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:46.778439 20467 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.726380 20526 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:04:46.834547 20322 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:46.785619 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58109.1us
I20260709 13:04:46.786237 20217 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:46.787051 15841 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:46.735029 20378 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:46.838501 20473 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.883613 20353 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.885483 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34666 (request call id 41) took 1287 ms. Trace:
I20260709 13:04:46.865109 20324 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.732882 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.885881 15082 rpcz_store.cc:276] 0709 13:04:45.595752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.595906 (+   154us) service_pool.cc:224] Handling call
0709 13:04:46.882610 (+1286704us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b54001c5820. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f961520 00007f0d3f9de652 00007f0d449ac60a 00007f0d3df327d4 00007f0d3df32762 00007f0d3df32691 00007f0d3df3260c 00007f0d3df318c2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:46.882873 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195442048}
I20260709 13:04:46.888196 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:46.884361 15085 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:46.790011 20335 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.721040 19780 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.794345 20459 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.795060 20509 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.721287 20140 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:46.795578 20209 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:46.797735 20285 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.800429 20053 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.709738 20364 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.809875 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.809973 15237 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:46.811183 15234 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:46.830442 20481 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:46.858536 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.863144 15236 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:46.866767 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.919588 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1157) took 2093 ms. Trace:
I20260709 13:04:46.921336 15844 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:46.862283 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.920221 15155 rpcz_store.cc:276] 0709 13:04:44.826281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.876386 (+ 50105us) service_pool.cc:224] Handling call
0709 13:04:44.914419 (+ 38033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.914651 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.914684 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.955386 (+ 40702us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.311323 (+355937us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:45.311373 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.311440 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.743562 (+1432122us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:46.903123 (+159561us) raft_consensus.cc:1685] finished
0709 13:04:46.903145 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.919500 (+ 16355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11315072,"thread_start_us":297810,"threads_started":1,"wal-append.queue_time_us":731715}
W20260709 13:04:46.922367 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60160.8us
I20260709 13:04:46.894111 20324 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.830710 20429 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.833065 20184 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:46.784592 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.873385 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.882689 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.938580 15233 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:46.938935 15232 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:46.902769 20136 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:46.891968 15085 pending_rounds.cc:85] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 9
I20260709 13:04:46.942950 15085 pending_rounds.cc:107] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:46.943326 15085 raft_consensus.cc:2889] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.894992 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:46.944227 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49311.8us
I20260709 13:04:46.862808 20432 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.902326 20140 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:46.909410 20209 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:46.916039 15230 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:46.956756 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.957408 15230 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:46.924140 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:46.867686 15163 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:46.924674 20465 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:46.926142 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.926306 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.895450 20402 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:46.927608 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.929597 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.895684 19780 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:46.791129 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.973789 15235 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:46.972083 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.974396 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.974872 15230 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:46.941627 15228 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:46.902037 20379 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:46.905006 15840 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:46.976446 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:46.976979 20547 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:04:46.977075 15840 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:46.977682 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:46.977926 20548 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:04:46.902603 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.979419 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.980361 20437 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:46.928046 20322 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.981026 20437 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.981794 20322 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.955708 20053 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:46.963902 20209 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.986338 19997 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.987887 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.988588 15084 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:46.989121 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.989745 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.990386 15084 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:46.991355 20432 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:46.972496 20402 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:46.993546 20402 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.993924 20423 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:46.997424 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.998256 20525 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:46.998803 20525 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:46.976481 20429 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.999908 20525 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:46.946805 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:46.867501 20314 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:46.962241 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.964030 15229 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:46.965234 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.011025 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:46.970779 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.972028 15234 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:46.975365 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.001807 15080 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:46.963689 20140 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:46.998271 15236 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:46.948182 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:46.980541 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:46.974969 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:47.010061 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 63352.5us
I20260709 13:04:47.023618 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:46.966792 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.024679 15088 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:47.014355 15163 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:47.015719 15230 raft_consensus.cc:2410] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 11.
I20260709 13:04:47.016119 15840 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:47.021939 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.028972 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.029647 15163 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:47.029788 15236 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:47.030598 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.032246 15235 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:47.034255 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:47.034888 15228 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.038300 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:46.937194 20481 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.039832 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.041045 15233 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.042340 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.040638 15236 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:47.008196 20053 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:47.046326 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.044699 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.020268 20459 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:47.022143 20437 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:47.049342 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.064779 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.066347 15236 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:47.067505 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.068948 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.070444 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.071878 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.064010 15233 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:04:47.074729 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.075382 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.087360 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.093693 15231 raft_consensus.cc:2410] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:47.093802 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.011632 20328 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:04:47.041628 20525 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.095345 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:46.971324 20473 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.121454 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:47.122246 15155 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.123714 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:47.108160 20556 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:47.124472 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:47.135948 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.100318 20461 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:47.089231 20459 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:47.050168 20522 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:04:47.105254 20465 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:47.032604 20353 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.244299 15236 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:47.246846 15155 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:47.044916 20184 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.105094 20251 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 13:04:47.162241 20427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.011s
I20260709 13:04:47.260542 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.302572 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:47.163722 20408 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:47.158473 20045 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.305930 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:47.323410 15159 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:47.315182 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:47.324877 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.326646 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.233244 20465 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:47.332924 20276 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:47.326216 20427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.003s	sys 0.013s
I20260709 13:04:47.345146 20235 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:47.345121 20276 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:47.315073 20341 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:47.322830 20444 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:47.330940 20045 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:47.361738 20555 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:04:47.361831 20481 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.363454 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.304320 20557 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.372931 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:47.437594 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64784.8us
W20260709 13:04:47.438285 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.439308 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:47.440145 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:47.285176 20520 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.308074 20251 consensus_queue.cc:1243] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:47.360561 20550 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
I20260709 13:04:47.323824 20178 raft_consensus.cc:2764] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:04:47.353614 20552 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 13:04:47.341620 20184 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.341034 20465 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.333112 20423 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:47.369612 20554 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
I20260709 13:04:47.330240 20335 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:47.418998 20045 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:47.417686 15159 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:47.472339 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.473541 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:47.424546 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.426855 20342 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.383215 20276 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.430646 20314 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:47.390574 20444 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:47.455251 20432 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:47.347404 20364 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.269757 20053 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:47.449136 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.463444 15848 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:47.459820 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:47.463207 20509 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:47.443136 20429 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.510510 20429 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.431262 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:47.487577 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 392311us
I20260709 13:04:47.514642 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.455039 20557 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:47.503541 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.385300 20459 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.468891 20402 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.507393 20509 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.521864 15162 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:04:47.513092 20429 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.489558 20444 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.500151 20342 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:47.552902 15163 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:47.555058 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.555984 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:47.556212 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.686s (SENT) [suppressed 18 similar messages]
W20260709 13:04:47.537282 15088 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 13:04:47.499087 20314 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:47.515904 20360 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.000s
I20260709 13:04:47.543756 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2409) took 1129 ms. Trace:
I20260709 13:04:47.545043 20402 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.540829 20509 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.546989 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.547534 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:47.596488 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49064.7us
I20260709 13:04:47.597291 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.598469 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.549109 20381 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:47.549223 20477 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:47.550921 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.599633 20364 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:47.600116 20432 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:47.600999 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.614156 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:47.602623 20322 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.620s	user 0.004s	sys 0.007s
I20260709 13:04:47.558414 20429 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:47.614344 20477 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.620112 20477 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:47.621095 20364 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:47.622025 20324 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.679s	user 0.006s	sys 0.005s
I20260709 13:04:47.595209 20335 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:04:47.597381 15158 rpcz_store.cc:276] 0709 13:04:46.414384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.416059 (+  1675us) service_pool.cc:224] Handling call
0709 13:04:46.416441 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.428439 (+ 11998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.428474 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.460248 (+ 31774us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.747698 (+287450us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18002ce780. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:46.895525 (+147827us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:46.895566 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.895635 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.539553 (+643918us) raft_consensus.cc:1685] finished
0709 13:04:47.539572 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.543670 (+  4098us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25555,"raft.queue_time_us":4038,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5655,"spinlock_wait_cycles":8692480,"thread_start_us":147631,"threads_started":1,"wal-append.queue_time_us":579741}
I20260709 13:04:47.559579 20561 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:47.627876 20432 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:47.599139 15228 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.000s
I20260709 13:04:47.628517 15228 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:47.617141 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.561820 20401 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:47.562883 15088 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:47.621980 20364 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:47.635278 15236 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:04:47.629596 15083 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:47.610756 20314 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:47.639598 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1160) took 2289 ms. Trace:
I20260709 13:04:47.644179 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.617424 20429 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:47.640286 15154 rpcz_store.cc:276] 0709 13:04:45.343320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.343460 (+   140us) service_pool.cc:224] Handling call
0709 13:04:45.350195 (+  6735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.519093 (+168898us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:45.581190 (+ 62097us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.142639 (+561449us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.143253 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.143461 (+   208us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:46.143514 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.663312 (+519798us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b48002001d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:04:47.632333 (+969021us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54003e0b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:47.632406 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.632463 (+    57us) raft_consensus.cc:1685] finished
0709 13:04:47.632483 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.632640 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28904,"prepare.queue_time_us":62675,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":47476,"raft.run_cpu_time_us":3579,"raft.run_wall_time_us":739386,"spinlock_wait_cycles":428489728,"thread_start_us":499164,"threads_started":2,"wal-append.queue_time_us":400928,"wal-append.run_cpu_time_us":3880,"wal-append.run_wall_time_us":1353923}
I20260709 13:04:47.651998 15158 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:47.622557 20324 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.631366 20432 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.636906 20364 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:47.608217 20360 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:47.656458 15081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.022s	sys 0.034s
I20260709 13:04:47.636085 20188 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:47.650310 20209 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.664s	user 0.004s	sys 0.008s
W20260709 13:04:47.704870 15081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.023s	sys 0.035s
I20260709 13:04:47.628183 20561 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:47.670419 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 114216us
W20260709 13:04:47.655370 20466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.015s	sys 0.039s
W20260709 13:04:47.703297 15079 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.010s	sys 0.045s
I20260709 13:04:47.634788 20401 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:47.696575 20557 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.705529 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.654129 20429 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:47.653283 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 103014us
I20260709 13:04:47.654575 20314 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:47.715775 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 275670us
I20260709 13:04:47.717685 15232 raft_consensus.cc:2368] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
W20260709 13:04:47.711776 15079 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.010s	sys 0.046s
I20260709 13:04:47.718568 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:47.707273 20188 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:47.719890 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.703469 20364 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:47.721216 15839 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEARNER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:04:47.711381 20466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.016s	sys 0.040s
I20260709 13:04:47.709417 20251 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.728037 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.731562 15839 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
W20260709 13:04:47.727945 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 113630us
I20260709 13:04:47.733008 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.734090 15840 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:47.732493 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:47.737381 15841 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:04:47.717695 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:47.743008 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:47.714740 20401 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.715647 20568 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.708927 20209 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.761441 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.762773 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.763491 15083 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:47.763660 20381 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:47.764129 20381 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.764272 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.764950 20568 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.767199 20568 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:47.739145 20322 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:04:47.775748 20564 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.303s	user 0.000s	sys 0.002s
I20260709 13:04:47.777602 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.780000 20209 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:47.781142 15847 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:47.782428 20285 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.785187 20575 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.790548 15083 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:04:47.794049 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:47.794910 15083 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:47.752941 20560 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:47.719409 20314 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.802911 20360 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.803261 15084 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:47.803596 20360 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:47.804726 15084 raft_consensus.cc:1217] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Deduplicated request from leader. Original: 10.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 13:04:47.805271 15084 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:47.814059 20400 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.816191 15228 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:47.817607 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1129) took 1421 ms. Trace:
I20260709 13:04:47.799585 20188 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.819111 20364 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.819682 20188 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.818137 15237 rpcz_store.cc:276] 0709 13:04:46.395850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.396319 (+   469us) service_pool.cc:224] Handling call
0709 13:04:47.817432 (+1421113us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400176954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:47.817545 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432391040}
I20260709 13:04:47.828126 20560 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.828675 20560 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:47.830786 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.834625 20314 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:47.835924 20324 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.839123 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:47.841100 20289 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:47.841717 20289 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.842967 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2393) took 1102 ms. Trace:
I20260709 13:04:47.843366 15230 rpcz_store.cc:276] 0709 13:04:46.740267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.842183 (+1101916us) service_pool.cc:224] Handling call
0709 13:04:47.842507 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.842673 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.842697 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.842734 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:47.842764 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.842819 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.842913 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.844455 20532 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:47.844856 20289 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:47.845013 19997 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:47.850536 15230 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:04:47.816042 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2379) took 1079 ms. Trace:
I20260709 13:04:47.851557 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:47.851698 15236 rpcz_store.cc:276] 0709 13:04:46.736513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.315303 (+578790us) service_pool.cc:224] Handling call
0709 13:04:47.629313 (+314010us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:47.636980 (+  7667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.815838 (+178858us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400176450. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:47.815982 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721217920}
I20260709 13:04:47.852294 15083 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:47.853572 15236 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:47.853819 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2380) took 1117 ms. Trace:
I20260709 13:04:47.854394 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2396) took 1112 ms. Trace:
I20260709 13:04:47.854996 15236 rpcz_store.cc:276] 0709 13:04:46.742459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.852745 (+1110286us) service_pool.cc:224] Handling call
0709 13:04:47.853146 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.854061 (+   915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.854323 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.854337 15234 rpcz_store.cc:276] 0709 13:04:46.736874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.420385 (+683511us) service_pool.cc:224] Handling call
0709 13:04:47.420797 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.421209 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.421243 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.421765 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.705086 (+283321us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:47.853040 (+147954us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.853415 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.853511 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.853566 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:47.853585 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.853738 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79732,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":9095,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":427191,"threads_started":2,"wal-append.queue_time_us":330642}
I20260709 13:04:47.858878 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2135) took 1114 ms. Trace:
I20260709 13:04:47.859148 15234 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:47.859544 15236 rpcz_store.cc:276] 0709 13:04:46.744474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.856463 (+1111989us) service_pool.cc:224] Handling call
0709 13:04:47.858156 (+  1693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.858423 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.858463 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.858525 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.858581 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.858658 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.858807 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.862114 15236 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:04:47.862480 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.862910 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2142) took 1116 ms. Trace:
I20260709 13:04:47.863471 15236 rpcz_store.cc:276] 0709 13:04:46.746365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.861158 (+1114793us) service_pool.cc:224] Handling call
0709 13:04:47.861583 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.862584 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.862840 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.867698 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.870615 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.872814 15843 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:47.875713 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2395) took 1134 ms. Trace:
I20260709 13:04:47.876544 15230 rpcz_store.cc:276] 0709 13:04:46.742182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.844401 (+1102219us) service_pool.cc:224] Handling call
0709 13:04:47.844717 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.875367 (+ 30650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.875645 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.879887 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2137) took 1134 ms. Trace:
I20260709 13:04:47.880345 15234 rpcz_store.cc:276] 0709 13:04:46.745081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.857731 (+1112650us) service_pool.cc:224] Handling call
0709 13:04:47.858117 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.865029 (+  6912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.879817 (+ 14788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.884570 15845 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:47.884454 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1137) took 1039 ms. Trace:
I20260709 13:04:47.887423 15236 rpcz_store.cc:276] 0709 13:04:46.845383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.883597 (+1038214us) service_pool.cc:224] Handling call
0709 13:04:47.883912 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.884118 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.884144 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.884187 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:47.884220 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.884276 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.884393 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.828105 19780 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.810s	user 0.004s	sys 0.004s
W20260709 13:04:47.828105 20140 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.788s	user 0.006s	sys 0.005s
I20260709 13:04:47.797032 20580 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:47.881438 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.851366 20532 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.880115 20235 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:47.880414 20577 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:47.883903 20525 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.770s	user 0.004s	sys 0.007s
I20260709 13:04:47.899748 20235 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:47.902864 20235 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.904119 20235 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.910919 20560 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.914590 15081 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:47.794529 20575 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.894963 20580 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.897909 20532 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:47.931008 15080 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:47.933779 15080 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:47.936051 15237 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:04:47.936563 15237 pending_rounds.cc:85] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:04:47.936882 15237 pending_rounds.cc:107] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:47.937184 15237 raft_consensus.cc:2889] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.938203 15085 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:47.938791 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.940801 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 197926us
I20260709 13:04:47.943426 15236 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:47.949245 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.896489 19780 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.954035 15234 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:04:47.958350 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1280) took 1559 ms. Trace:
I20260709 13:04:47.958699 15848 rpcz_store.cc:276] 0709 13:04:46.398832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.757566 (+358734us) service_pool.cc:224] Handling call
0709 13:04:46.757880 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.506501 (+748621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.958144 (+451643us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54004a2754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:47.958298 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622181632}
I20260709 13:04:47.896752 20140 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.955096 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:47.961831 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 203068us
I20260709 13:04:47.924980 20580 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.971045 15161 raft_consensus.cc:1217] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 13:04:47.971544 15161 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:47.971927 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.979637 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.981419 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.983392 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.986552 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:47.990554 15082 raft_consensus.cc:2368] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:47.916985 20577 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.956770 20432 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.996565 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:47.994890 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:47.961457 20353 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:47.998217 15162 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:47.960649 20561 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:47.931104 20276 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:47.954433 20557 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:47.954243 20525 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:47.988953 20520 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.078189 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2213) took 1104 ms. Trace:
W20260709 13:04:48.094987 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.087594 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2823) took 1118 ms. Trace:
W20260709 13:04:48.130537 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 135857us
I20260709 13:04:48.060515 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2383) took 1322 ms. Trace:
I20260709 13:04:48.017827 15163 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:48.052332 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.194043 15085 rpcz_store.cc:276] 0709 13:04:46.973793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.054210 (+ 80417us) service_pool.cc:224] Handling call
0709 13:04:47.054506 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.955034 (+900528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.077840 (+122806us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54000db6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.078111 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031740800}
I20260709 13:04:48.065464 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2140) took 1319 ms. Trace:
I20260709 13:04:48.038149 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.199236 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:48.044526 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.198556 15233 rpcz_store.cc:276] 0709 13:04:46.745868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.947913 (+202045us) service_pool.cc:224] Handling call
0709 13:04:48.065196 (+1117283us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400319054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:48.065375 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41079680}
W20260709 13:04:48.111996 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.201503 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.106894 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1281) took 1707 ms. Trace:
I20260709 13:04:48.202728 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.190294 15083 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:04:48.204010 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2830) took 1055 ms. Trace:
I20260709 13:04:48.204128 15080 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:48.094338 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2384) took 1356 ms. Trace:
I20260709 13:04:48.199498 15848 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:48.204409 15083 rpcz_store.cc:276] 0709 13:04:47.148222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.851283 (+703061us) service_pool.cc:224] Handling call
0709 13:04:48.203950 (+352667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:48.058110 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.205852 15839 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:48.057698 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.060073 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.206976 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1335) took 1622 ms. Trace:
I20260709 13:04:48.195533 15082 rpcz_store.cc:276] 0709 13:04:46.969085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.977395 (+  8310us) service_pool.cc:224] Handling call
0709 13:04:48.087281 (+1109886us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540014e454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:48.087509 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2327704832}
I20260709 13:04:48.204927 15228 rpcz_store.cc:276] 0709 13:04:46.737865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.636502 (+898637us) service_pool.cc:224] Handling call
0709 13:04:47.639274 (+  2772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.816685 (+177411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.080594 (+263909us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400319a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.094262 (+ 13668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406061440}
I20260709 13:04:48.225633 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.224766 15843 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:48.025885 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:48.233215 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 119567us
I20260709 13:04:48.170672 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:48.202569 15847 rpcz_store.cc:276] 0709 13:04:46.399149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.785460 (+386311us) service_pool.cc:224] Handling call
0709 13:04:46.892034 (+106574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.781874 (+889840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.105630 (+323756us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54001e0fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.106609 (+   979us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001e0fd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:48.106804 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2288431744}
I20260709 13:04:48.198160 15846 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:04:48.256124 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.227429 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.277300 15839 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.277401 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:48.278460 15085 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:48.279176 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:48.196893 15235 rpcz_store.cc:276] 0709 13:04:46.737623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.516504 (+778881us) service_pool.cc:224] Handling call
0709 13:04:47.517117 (+   613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.517600 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:47.711413 (+193813us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.964436 (+253023us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.989349 (+ 24913us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.989531 (+   182us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:47.996368 (+  6837us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.996561 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.996633 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.060236 (+ 63603us) raft_consensus.cc:1685] finished
0709 13:04:48.060266 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.060440 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22769,"prepare.queue_time_us":266591,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":116,"raft.queue_time_us":621,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":29463,"thread_start_us":412028,"threads_started":2,"wal-append.queue_time_us":338392}
W20260709 13:04:48.281082 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 110572us
I20260709 13:04:48.282128 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:48.207456 15844 rpcz_store.cc:276] 0709 13:04:46.584934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.005252 (+420318us) service_pool.cc:224] Handling call
0709 13:04:47.005608 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.005818 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:47.039276 (+ 33458us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.785498 (+746222us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b1800401c20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:04:47.998359 (+212861us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.998414 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:47.998476 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.998551 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.206899 (+208348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4747,"prepare.queue_time_us":32287,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":34,"raft.queue_time_us":94,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":9596672,"thread_start_us":212623,"threads_started":1,"wal-append.queue_time_us":1069725}
I20260709 13:04:48.284915 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.289186 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2412) took 1845 ms. Trace:
I20260709 13:04:48.289880 15156 rpcz_store.cc:276] 0709 13:04:46.443302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.443519 (+   217us) service_pool.cc:224] Handling call
0709 13:04:46.443788 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.671084 (+227296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.671119 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:46.671187 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:46.671344 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.288455 (+1617111us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540017d754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.288600 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.289028 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521907072}
I20260709 13:04:48.025276 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:47.989259 20364 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:48.075042 20563 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.522s	user 0.000s	sys 0.002s
I20260709 13:04:47.991992 20526 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:48.385552 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.992498 20522 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:48.392498 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 110472us
I20260709 13:04:48.401331 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:48.402122 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.402408 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.402725 15845 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:48.402909 15847 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:48.395829 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:47.997589 20456 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:48.400035 15157 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:48.390385 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 156117us
I20260709 13:04:48.062204 20461 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:48.048413 20251 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:48.037724 20560 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.395784 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:47.995548 20314 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.012445 20400 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.426s
I20260709 13:04:48.054875 20509 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:47.998838 20386 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.440s
I20260709 13:04:48.057468 20444 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.076548 20136 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:48.087344 20575 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:48.082587 19780 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:48.038743 20553 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.893s	user 0.000s	sys 0.004s
I20260709 13:04:48.280752 20328 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.560070 15839 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:48.414032 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:48.073784 20569 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.510s	user 0.003s	sys 0.000s
W20260709 13:04:48.511394 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 494101us
I20260709 13:04:48.629300 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:48.471063 20184 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.001s
I20260709 13:04:48.577987 15843 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:48.556691 15848 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:48.295678 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:48.092212 20379 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:48.224489 19997 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:48.594576 15082 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:48.620402 15083 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.639246 15159 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:48.418336 20461 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:48.004158 20360 raft_consensus.cc:2764] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:48.053309 20140 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.390045 20378 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:48.658114 20378 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:48.658741 20378 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.659632 20140 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:48.660255 20532 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.002s
W20260709 13:04:48.487936 20465 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.000s	user 0.008s	sys 0.004s
I20260709 13:04:48.661126 20045 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.029769 20408 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.606s
I20260709 13:04:48.661456 20465 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.662046 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2207) took 1743 ms. Trace:
I20260709 13:04:48.661717 20045 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.484467 20364 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:48.462016 20432 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.640817 20379 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:48.662434 15086 rpcz_store.cc:276] 0709 13:04:46.918688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.920961 (+  2273us) service_pool.cc:224] Handling call
0709 13:04:46.947215 (+ 26254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.013998 (+ 66783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.014041 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.055323 (+ 41282us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.460552 (+405229us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:47.460589 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.460655 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.072936 (+612281us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:48.661739 (+588803us) raft_consensus.cc:1685] finished
0709 13:04:48.661776 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.661987 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54937,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"spinlock_wait_cycles":315984256,"thread_start_us":405456,"threads_started":2,"wal-append.queue_time_us":419603}
I20260709 13:04:48.664084 20045 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:48.663858 20432 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.665567 15085 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:48.412307 20289 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:48.047605 20401 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.636610 15088 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:48.486217 20276 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:48.624044 15232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2397) took 1880 ms (client timeout 1956 ms). Trace:
W20260709 13:04:48.603266 20473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.010s	sys 0.022s
I20260709 13:04:48.488749 20525 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 13:04:48.461737 20322 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:48.660742 20532 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.690232 20322 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.691949 20532 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.692313 20322 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.409648 20456 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:48.640473 20053 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:48.638687 20178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.072s	sys 0.077s
I20260709 13:04:48.469937 20353 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:48.292542 20382 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.343s
I20260709 13:04:48.646929 20444 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:48.646931 20285 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.864s	user 0.007s	sys 0.003s
I20260709 13:04:48.662786 20364 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.676568 20289 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:48.628515 20402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.014s	sys 0.026s
I20260709 13:04:48.726583 15842 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:48.678696 15845 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:48.679154 15229 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:48.683393 15847 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:48.578310 20136 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:48.656741 20235 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.752s	user 0.005s	sys 0.004s
I20260709 13:04:48.662289 20408 consensus_queue.cc:1243] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:48.730983 20364 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.731956 20383 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.686535 15232 rpcz_store.cc:269] 0709 13:04:46.743468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.945253 (+201785us) service_pool.cc:224] Handling call
0709 13:04:48.588564 (+1643311us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b5400176e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:48.623930 (+ 35366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673710976}
I20260709 13:04:48.641358 20526 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.733556 20526 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.735776 20526 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.693437 15084 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:48.685945 20381 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.921s	user 0.013s	sys 0.000s
I20260709 13:04:48.736929 20289 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:48.737206 20381 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.738611 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2209) took 1806 ms. Trace:
I20260709 13:04:48.739382 20381 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.595165 20314 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.664731 20045 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:48.692698 20401 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.717795 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2420) took 1641 ms. Trace:
I20260709 13:04:48.627223 20577 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:48.704550 20178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.072s	sys 0.077s
W20260709 13:04:48.688275 20473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.011s	sys 0.022s
I20260709 13:04:48.749766 20473 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.750241 20473 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.752308 20473 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.674834 15083 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:48.055840 20429 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.754462 20429 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.755273 15087 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:48.678539 20584 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.732491 20383 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:48.743750 20402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.015s	sys 0.026s
I20260709 13:04:48.686211 19997 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:48.738992 15079 rpcz_store.cc:276] 0709 13:04:46.932042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.932199 (+   157us) service_pool.cc:224] Handling call
0709 13:04:46.932521 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.932886 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:47.719143 (+786257us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.959551 (+240408us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.988704 (+ 29153us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.988884 (+   180us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:47.988920 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.988991 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.738395 (+749404us) raft_consensus.cc:1685] finished
0709 13:04:48.738417 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.738554 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27129,"prepare.queue_time_us":183537,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":10495,"raft.queue_time_us":3095,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":29810,"thread_start_us":916605,"threads_started":2,"wal-append.queue_time_us":772788}
I20260709 13:04:48.704444 20456 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.767025 20314 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:48.702658 20053 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.768332 20053 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:48.743163 15162 rpcz_store.cc:276] 0709 13:04:47.077506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.538818 (+461312us) service_pool.cc:224] Handling call
0709 13:04:47.539143 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.209139 (+669996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.717525 (+508386us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.717731 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055293696}
I20260709 13:04:48.747885 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:48.772399 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:48.775468 20401 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:48.776156 20401 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:48.776620 20401 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:48.777491 20401 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.634096 20328 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.739404 20509 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:48.765241 19997 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:48.720801 20353 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.788908 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:48.806567 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:48.789023 20594 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:04:48.790936 20465 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.792047 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2391) took 2052 ms. Trace:
I20260709 13:04:48.786010 20178 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:48.787421 15079 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:48.803560 20381 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.727273 20461 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.793638 20235 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:48.809988 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:48.817896 15083 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:48.792723 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1295) took 1373 ms. Trace:
I20260709 13:04:48.819483 15081 rpcz_store.cc:276] 0709 13:04:47.419753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.777321 (+357568us) service_pool.cc:224] Handling call
0709 13:04:48.792544 (+1015223us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540011add4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:48.792668 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312806272}
I20260709 13:04:48.810057 15237 rpcz_store.cc:276] 0709 13:04:46.739693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.825993 (+1086300us) service_pool.cc:224] Handling call
0709 13:04:47.826444 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.937745 (+111301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.791813 (+854068us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540005bc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.791987 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250917760}
W20260709 13:04:48.810626 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.035s	sys 0.054s
W20260709 13:04:48.821010 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.035s	sys 0.054s
W20260709 13:04:48.821394 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 624752us
W20260709 13:04:48.821926 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 233129us
I20260709 13:04:48.813604 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:48.809707 20314 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:48.761483 20584 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.786775 20382 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:04:48.835973 20580 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.000s
I20260709 13:04:48.832441 20045 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:48.822508 19997 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.833056 20314 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.833804 20432 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.805684 20378 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.835170 20289 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.835845 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:48.861436 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.862013 15162 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:48.862095 15237 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:48.826675 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.864354 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.864969 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.866878 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.817379 20381 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.867476 15237 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:48.868916 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.825265 20402 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:04:48.868024 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:48.794064 20136 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.841897 20444 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:48.883486 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.856422 20509 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:48.893586 20289 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.893252 20509 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.878839 19997 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:48.896212 20509 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.858844 20580 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.899416 20580 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.900147 20379 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.902652 20328 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.906109 20378 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:48.907464 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.908599 15232 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:48.859753 20360 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.909822 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.910094 20360 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.911252 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.914417 20314 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.859521 20364 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:48.915493 20364 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.889447 20045 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:48.923671 19780 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:48.924307 19780 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:48.925597 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:48.925522 19780 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.926537 15156 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:48.927524 19780 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.853201 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:48.855434 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:48.857354 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 383833us
W20260709 13:04:48.822300 20423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.015s	sys 0.026s
I20260709 13:04:48.870788 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.843834 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:48.873430 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.814653 20178 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.875058 20532 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.869585 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:48.879050 15155 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:04:48.949321 15155 pending_rounds.cc:85] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 10
W20260709 13:04:48.949200 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105432us
I20260709 13:04:48.949604 15155 pending_rounds.cc:107] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:04:48.949870 15155 raft_consensus.cc:2889] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.950970 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:48.823452 20235 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:48.927805 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:48.957598 15156 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:48.929497 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2388) took 2190 ms. Trace:
I20260709 13:04:48.930007 20053 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.934595 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:48.935441 20444 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:48.937119 20341 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:04:48.853104 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.943722 20314 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:48.900548 15236 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:48.940454 15232 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:48.943962 15234 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:48.981750 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.872421 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.983007 15235 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:48.942986 15162 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:48.983278 15233 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:48.983407 15155 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:48.947965 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:48.876796 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.985235 15237 raft_consensus.cc:2368] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:48.986114 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:48.948604 20423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.015s	sys 0.027s
I20260709 13:04:48.939219 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2153) took 1980 ms. Trace:
I20260709 13:04:48.979748 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:48.981801 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.972185 15154 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:48.990599 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.991044 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:48.991215 15232 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.951171 20526 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:48.958074 15231 rpcz_store.cc:276] 0709 13:04:46.738879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.703000 (+964121us) service_pool.cc:224] Handling call
0709 13:04:47.716368 (+ 13368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.833395 (+117027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.833432 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.833484 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:47.833526 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.929213 (+1095687us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400175f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:48.929277 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.929430 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268578176}
I20260709 13:04:48.950322 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:48.989830 15230 rpcz_store.cc:276] 0709 13:04:46.959171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.877855 (+918684us) service_pool.cc:224] Handling call
0709 13:04:47.878312 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.878771 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.878814 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.896497 (+ 17683us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.992493 (+ 95996us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:48.389815 (+397322us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.855643 (+465828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.855747 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.939005 (+ 83258us) raft_consensus.cc:1685] finished
0709 13:04:48.939027 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.939156 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":274544,"prepare.queue_time_us":397589,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":25559,"raft.queue_time_us":106318,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":13101,"thread_start_us":469747,"threads_started":2,"wal-append.queue_time_us":581824}
I20260709 13:04:48.988595 15160 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:48.992564 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:48.993315 15231 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:48.950778 20584 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:48.993778 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43539.6us
I20260709 13:04:48.972894 20364 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:48.999137 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:48.999704 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.978344 20444 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:49.000942 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:48.992377 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.000627 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:49.006171 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:49.007745 20487 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:49.008287 20487 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:49.009651 20585 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:49.009387 20487 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.010231 20585 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:49.012918 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.014204 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.005101 15234 raft_consensus.cc:2376] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15. Re-sending same reply.
I20260709 13:04:48.956676 20402 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:04:48.960824 20532 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:48.988245 20360 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:48.993475 20509 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:49.025533 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.026274 15231 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:48.989194 20314 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.037684 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:49.023455 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.015s	sys 0.020s
I20260709 13:04:49.038223 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:04:49.041018 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.016s	sys 0.020s
I20260709 13:04:49.018227 20364 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:04:49.041819 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 640563us
I20260709 13:04:49.031826 20584 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.033701 20335 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.043279 15843 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:49.043287 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.024209 20525 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:04:49.042452 15845 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:49.005005 20601 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:49.011264 20585 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.054211 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.058588 15845 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:49.057107 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.057556 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.039129 20360 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.065318 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.067607 15232 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:49.042593 20509 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.047034 20608 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.073562 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.058920 20381 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:49.066079 20557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.018s	sys 0.042s
I20260709 13:04:49.061455 20603 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:49.060067 20444 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.060043 20528 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:49.081207 20603 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:49.082050 20444 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.082099 20603 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.087961 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.088696 15080 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:49.090709 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.097079 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.096657 15154 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:49.099520 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.059182 20335 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.101752 15154 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:49.095947 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.105638 20528 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:49.106344 20178 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.107384 20053 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.108894 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.109500 20045 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:49.111225 20045 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:49.112231 20520 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.832s	user 0.011s	sys 0.000s
I20260709 13:04:49.079022 20378 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:04:49.112762 20285 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:49.113646 20378 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:49.114025 20378 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:49.077910 20608 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.114606 20509 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.114907 20378 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.116410 20378 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.118093 20314 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.079598 20584 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:49.092310 20383 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.119910 20335 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.121026 20235 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:49.122315 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.123633 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.125159 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.092689 20456 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.126794 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.127440 20383 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.128147 20383 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:49.129094 20383 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:49.129472 20383 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:49.128563 20612 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:49.130254 20383 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.131114 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.135342 20561 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:49.136505 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.136951 20561 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.137738 20612 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:49.137944 20360 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:49.138595 20360 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.139361 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.139770 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.140398 15083 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:49.087456 20557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.018s	sys 0.043s
I20260709 13:04:49.140937 20360 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.141712 20360 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:49.142235 20584 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.117995 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.140486 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.143504 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.141585 20276 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:49.116880 20608 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:49.153790 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.106824 20528 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.151216 15231 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:49.125613 15086 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.141027 20432 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:49.098147 15080 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:49.167400 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.112651 20520 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.156147 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.157822 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.161651 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:49.131544 20383 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:49.172794 15083 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:49.173013 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.175012 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.176952 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.177903 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.178979 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.125644 20532 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.128654 20456 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.181588 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.182447 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.182771 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.184065 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:49.169076 20025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.012s
I20260709 13:04:49.182996 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.186493 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.188594 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.191465 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.187556 15839 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:49.162798 20364 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.157047 20314 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:49.190199 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2864) took 1829 ms (client timeout 1787 ms). Trace:
I20260709 13:04:49.190366 15161 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:49.156119 20459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.610s	user 0.013s	sys 0.010s
I20260709 13:04:49.192852 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.195412 15841 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:49.198626 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.203466 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.204377 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.207870 15154 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:49.180433 20604 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:49.181411 20045 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.182235 20360 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:49.209932 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:49.205252 15157 rpcz_store.cc:269] 0709 13:04:47.360554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.424269 (+ 63715us) service_pool.cc:224] Handling call
0709 13:04:49.189990 (+1765721us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b540017e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.190131 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116999936}
I20260709 13:04:49.185959 20605 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:49.217561 15232 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:49.226415 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.211328 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.227427 15157 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:49.211117 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.233705 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.234885 15841 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:49.235278 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:49.212568 20459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.667s	user 0.013s	sys 0.010s
I20260709 13:04:49.169003 20432 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:49.181260 20437 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.213945 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:49.190997 20025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.013s	sys 0.013s
I20260709 13:04:49.214632 15844 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:49.226619 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.238220 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.251047 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.251725 15846 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.212059 15086 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:49.244395 15157 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:49.255481 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.259526 15154 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:04:49.225647 20360 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.268512 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.236950 20520 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:49.275808 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.278642 19780 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.279126 19780 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.248592 20432 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.252566 20604 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.281517 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.252867 20605 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.282860 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.295154 19780 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.296690 20607 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:49.298678 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.301868 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.272967 20360 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:49.306855 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.238797 20045 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.317308 19780 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:49.308776 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.317766 19780 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:49.318189 19780 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:49.319034 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.316386 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.306855 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.313773 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.322780 15051 leader_election.cc:400] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:49.323066 15051 leader_election.cc:403] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:49.327287 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.285691 20561 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.256536 20437 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.335397 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.370297 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.366211 15846 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
W20260709 13:04:49.367111 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44463.4us
I20260709 13:04:49.314153 20053 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.242172 20584 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.290617 20381 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:04:49.371646 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.394789 15086 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:49.394722 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2223) took 1536 ms. Trace:
I20260709 13:04:49.303915 20487 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:04:49.319034 20251 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:49.286368 20520 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.303462 20459 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:04:49.395888 15082 rpcz_store.cc:276] 0709 13:04:47.857868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.406671 (+548803us) service_pool.cc:224] Handling call
0709 13:04:48.406996 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.639260 (+232264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.394442 (+755182us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54002378d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.394655 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430775680}
I20260709 13:04:49.333230 20465 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:49.376047 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.369417 20577 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.371366 20509 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:49.466044 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.477556 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2417) took 2430 ms. Trace:
W20260709 13:04:49.488394 15088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2832) took 2183 ms (client timeout 1774 ms). Trace:
I20260709 13:04:49.300892 20335 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:04:49.366786 20285 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:49.373438 20568 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13.
I20260709 13:04:49.372045 20580 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.488727 15159 rpcz_store.cc:276] 0709 13:04:47.047557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.474436 (+426879us) service_pool.cc:224] Handling call
0709 13:04:47.474805 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.639683 (+1164878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.477260 (+837577us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001d8154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.477484 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677544320}
I20260709 13:04:49.485625 15237 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
W20260709 13:04:49.488880 15088 rpcz_store.cc:269] 0709 13:04:47.304810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.643908 (+339098us) service_pool.cc:224] Handling call
0709 13:04:49.488168 (+1844260us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540011a8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.488321 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279611264}
I20260709 13:04:49.495920 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.366145 20429 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:04:49.380879 20466 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.321029 20603 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.504570 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:49.507495 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 185502us
W20260709 13:04:49.514470 20620 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:04:49.514931 20603 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.515770 20603 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.483037 20251 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:49.482517 20053 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:49.370388 20561 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.485291 20465 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:49.531030 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 159455us
W20260709 13:04:49.490463 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 155169us
I20260709 13:04:49.535056 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.462970 19780 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.494033 20314 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.506707 20459 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:49.516413 20603 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.522579 20251 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:49.572877 19780 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.567826 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:49.522605 20606 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.541357 20465 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:49.578629 20644 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:49.580113 20623 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:49.548802 20580 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:49.517125 20466 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.540506 20053 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:49.632251 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64526.4us
I20260709 13:04:49.494017 20605 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.633133 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.593021 20509 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.565356 20314 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.595649 20580 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:49.522365 20604 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:49.680454 20570 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 2.116s	user 0.019s	sys 0.017s
I20260709 13:04:49.636128 20364 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:49.698491 15234 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.673256 20639 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:49.707358 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74313.2us
I20260709 13:04:49.709335 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.710412 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.711038 15082 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.602746 20644 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:49.746691 20616 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:04:49.749630 20459 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:49.751065 20624 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:04:49.698899 20045 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.723719 20580 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.753088 20603 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.753407 20580 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:49.705842 20626 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:04:49.726054 20617 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 13:04:49.725731 20628 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
W20260709 13:04:49.726763 20619 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:49.759147 20053 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.760474 20561 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.726375 20646 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.729142 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.761188 20646 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.731237 15082 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:49.761577 20646 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.732872 20466 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:49.727608 20621 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.209s	user 0.004s	sys 0.000s
I20260709 13:04:49.712095 20364 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.742861 20509 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.717898 20437 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.711536 20487 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.724108 20639 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:49.769604 20639 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.733438 20520 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.775265 20401 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Committing config change with OpId 10.8: config changed from index 5 to 8, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:49.777695 20314 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.777979 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2222) took 1920 ms. Trace:
I20260709 13:04:49.778347 15085 rpcz_store.cc:276] 0709 13:04:47.857550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.395961 (+538411us) service_pool.cc:224] Handling call
0709 13:04:48.396252 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.676294 (+280042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.777752 (+1101458us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540039b554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.777923 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618975488}
I20260709 13:04:49.779850 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.777154 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.776105 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.782182 20461 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:49.782575 20379 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.783638 15841 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:49.784806 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1145) took 1760 ms. Trace:
I20260709 13:04:49.785210 15236 rpcz_store.cc:276] 0709 13:04:48.024393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.025588 (+  1195us) service_pool.cc:224] Handling call
0709 13:04:49.784613 (+1759025us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400318b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.784735 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532641664}
I20260709 13:04:49.785537 20644 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.786347 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:49.787580 20487 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.787743 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.788095 20487 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.789151 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.790369 20487 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:49.791935 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.792352 20364 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.792801 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.794492 15843 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:49.794687 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:49.795712 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.796330 15840 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
W20260709 13:04:49.797899 20610 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.492s	user 0.002s	sys 0.000s
I20260709 13:04:49.804695 20353 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.805590 20639 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.806198 20639 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.807796 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2218) took 2757 ms. Trace:
I20260709 13:04:49.808152 15087 rpcz_store.cc:276] 0709 13:04:47.050439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.837080 (+786641us) service_pool.cc:224] Handling call
0709 13:04:47.837392 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.755755 (+918363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.807570 (+1051815us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.807729 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110486784}
W20260709 13:04:49.809557 20378 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.002s
W20260709 13:04:49.816370 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.818661 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.822475 20140 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.824132 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.824999 20140 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.826615 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.827390 20444 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.827963 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.830363 20609 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:49.830962 20609 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:49.831264 20444 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:49.819772 20432 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 13:04:49.833194 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1279) took 3434 ms. Trace:
I20260709 13:04:49.832975 20432 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.833549 15842 rpcz_store.cc:276] 0709 13:04:46.398530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.714015 (+315485us) service_pool.cc:224] Handling call
0709 13:04:46.714319 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.727131 (+2012812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.832915 (+1105784us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400580054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.833108 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4626783104}
I20260709 13:04:49.820315 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:49.835848 20557 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.835929 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2382) took 3098 ms. Trace:
I20260709 13:04:49.836314 20557 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.836350 15229 rpcz_store.cc:276] 0709 13:04:46.737396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.423062 (+685666us) service_pool.cc:224] Handling call
0709 13:04:48.074242 (+651180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.728079 (+653837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.834950 (+1106871us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400319054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.835754 (+   804us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b5400319050. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.835877 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887468160}
I20260709 13:04:49.837492 20444 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:49.837800 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.837868 15846 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:49.838016 20444 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.839252 20557 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.839707 20532 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.840306 20444 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.820060 20459 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.842716 20459 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.853931 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.854768 20615 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:49.855196 20615 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:49.855556 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.856029 20615 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:49.856815 20383 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 13:04:49.820866 20276 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:49.860648 20276 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:49.861078 20276 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:49.860989 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.861888 15228 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 10: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:49.862828 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.864382 15843 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:49.865368 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2217) took 2818 ms. Trace:
I20260709 13:04:49.865671 15845 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:49.865804 15228 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:49.867817 20615 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.869228 20423 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:49.869681 20423 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:49.870563 20423 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.871824 20423 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:49.872182 20465 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.856945 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.938908 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.874805 20136 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:49.950309 15087 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:49.821487 20520 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.877888 15228 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 13:04:49.865746 15084 rpcz_store.cc:276] 0709 13:04:47.047484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.805961 (+758477us) service_pool.cc:224] Handling call
0709 13:04:47.806266 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.736348 (+930082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.865181 (+1128833us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:49.865316 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2039905152}
I20260709 13:04:49.862069 20276 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:49.824144 20360 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.003s
I20260709 13:04:49.863067 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:49.883699 20353 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:49.883677 20432 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:49.884104 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:49.978550 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 94525.1us
I20260709 13:04:49.980105 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:49.885941 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:49.886363 15156 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:49.981046 15156 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:49.887050 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2407) took 1014 ms. Trace:
I20260709 13:04:49.888084 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:49.891057 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:49.982281 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 94271.1us
I20260709 13:04:49.891784 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:49.894941 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.983983 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.897056 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1188) took 1030 ms. Trace:
I20260709 13:04:49.982609 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:49.898238 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.986513 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:49.838522 15229 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:49.827337 20251 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:49.984656 15161 rpcz_store.cc:276] 0709 13:04:48.866581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.868583 (+  2002us) service_pool.cc:224] Handling call
0709 13:04:49.190190 (+321607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.205421 (+ 15231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.881190 (+675769us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b540017e150. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.896984 (+ 15794us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738711424}
I20260709 13:04:49.916632 20532 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:04:49.913411 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:50.012265 15086 raft_consensus.cc:2368] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:49.913976 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:49.913794 20646 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:49.914141 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:50.044781 15155 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:49.919071 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:49.920370 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:49.922730 15079 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:49.880607 15234 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
W20260709 13:04:49.940420 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 356628us
I20260709 13:04:49.939049 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:49.974193 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.048826 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.981489 20456 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:49.981926 15233 rpcz_store.cc:276] 0709 13:04:48.872641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.873136 (+   495us) service_pool.cc:224] Handling call
0709 13:04:49.886871 (+1013735us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400058cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:49.886998 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88485760}
I20260709 13:04:50.052537 15233 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:04:50.055130 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 116148us
W20260709 13:04:49.911233 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 83320us
I20260709 13:04:49.994303 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:50.057789 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63596.6us
I20260709 13:04:50.058771 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:49.992575 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:50.009702 20209 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:49.991302 20360 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.015779 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:50.023136 15083 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:50.024238 20251 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:50.025758 20487 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:49.950426 20423 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:04:50.027088 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:50.027527 20429 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:50.028370 15084 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:50.028553 15080 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:50.029304 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:50.031227 15229 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:04:49.917112 20444 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 13:04:50.083997 15228 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:04:50.084872 15229 pending_rounds.cc:85] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:04:50.085243 15229 pending_rounds.cc:107] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:04:50.085530 15229 raft_consensus.cc:2889] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.081140 15231 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:50.039548 20609 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.042810 20276 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:49.992888 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:50.046511 15235 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:50.056548 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:50.060755 15161 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:04:50.099661 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:50.069199 20209 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:50.083523 20487 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:50.154872 20444 consensus_queue.cc:1243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:50.081771 20360 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:50.070242 20608 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:50.129796 15843 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:49.951608 20378 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:04:50.078889 20466 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:50.219303 15845 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:50.079562 20353 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:50.068957 20437 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.219470 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1155) took 1033 ms. Trace:
I20260709 13:04:50.254140 15231 rpcz_store.cc:276] 0709 13:04:49.186063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.186225 (+   162us) service_pool.cc:224] Handling call
0709 13:04:50.166837 (+980612us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54003d9ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:50.218573 (+ 51736us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045745920}
I20260709 13:04:50.099920 20386 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:50.225725 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 125080us
I20260709 13:04:50.234790 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:50.235587 20209 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.239764 15844 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:50.242915 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:50.257109 15839 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:50.220220 15846 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 5 to 8, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:50.249737 20557 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.270123 20557 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:50.246351 20608 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:50.277621 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.019s	sys 0.011s
W20260709 13:04:50.278124 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.020s	sys 0.011s
W20260709 13:04:50.278802 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 743839us
I20260709 13:04:50.278985 15839 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:50.279825 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:50.246999 20609 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:50.251291 20353 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:50.304530 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:50.261382 20285 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:04:50.283043 20584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.038s
I20260709 13:04:50.277724 20608 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:50.310878 20584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.038s
I20260709 13:04:50.259639 20437 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.294993 20353 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:50.279135 20487 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:50.327068 20401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.021s	sys 0.044s
W20260709 13:04:50.318076 20604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.009s	sys 0.024s
I20260709 13:04:50.309432 20653 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:50.376498 20604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.024s
I20260709 13:04:50.353093 20646 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:50.429674 14981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 323) took 18321 ms (client timeout 15000 ms). Trace:
I20260709 13:04:50.016036 14985 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 13, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:04:50.069098 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53413.8us
I20260709 13:04:50.435842 14985 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:50.431030 14981 rpcz_store.cc:269] 0709 13:04:32.107914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.111328 (+  3414us) service_pool.cc:224] Handling call
0709 13:04:32.489900 (+378572us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:50.429157 (+17939257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:42.719872 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:42.732603 (+ 12731us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:42.732683 (+    80us) write_op.cc:435] Acquired schema lock
0709 13:04:42.732715 (+    32us) tablet.cc:662] Decoding operations
0709 13:04:42.802294 (+ 69579us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:42.802433 (+   139us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:42.802479 (+    46us) tablet.cc:685] Acquiring locks for 30 operations
0709 13:04:42.867845 (+ 65366us) tablet.cc:701] Row locks acquired
0709 13:04:42.886875 (+ 19030us) write_op.cc:260] PREPARE: finished
0709 13:04:42.912867 (+ 25992us) write_op.cc:270] Start()
0709 13:04:42.942473 (+ 29606us) write_op.cc:276] Timestamp: P: 1783602282907472 usec, L: 0
0709 13:04:42.942515 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:04:42.961312 (+ 18797us) log.cc:844] Serialized 11800 byte log entry
0709 13:04:43.448602 (+487290us) op_driver.cc:464] REPLICATION: finished
0709 13:04:44.125900 (+677298us) write_op.cc:301] APPLY: starting
0709 13:04:44.126036 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:45.464193 (+1338157us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:45.464326 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:46.140101 (+675775us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:46.140688 (+   587us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=0,mrs_lookups=0
0709 13:04:46.140765 (+    77us) write_op.cc:312] APPLY: finished
0709 13:04:46.342275 (+201510us) log.cc:844] Serialized 260 byte log entry
0709 13:04:46.748663 (+406388us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3c8da0ac 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b
0709 13:04:47.403680 (+655017us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:47.429142 (+ 25462us) write_op.cc:454] Released schema lock
0709 13:04:47.429252 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:47.429755 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:47.429784 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2796747,"reported_tablets":45,"spinlock_wait_cycles":421492864,"child_traces":[["op",{"apply.queue_time_us":493646,"apply.run_cpu_time_us":74494,"apply.run_wall_time_us":3456111,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129707,"num_ops":30,"prepare.queue_time_us":409881,"prepare.run_cpu_time_us":35327,"prepare.run_wall_time_us":439293,"raft.queue_time_us":157411,"raft.run_cpu_time_us":16988,"raft.run_wall_time_us":398933,"replication_time_us":458642,"spinlock_wait_cycles":70715136,"thread_start_us":1762897,"threads_started":5,"wal-append.queue_time_us":779485,"wal-append.run_cpu_time_us":13380,"wal-append.run_wall_time_us":2149569}]]}
I20260709 13:04:50.448046 15051 leader_election.cc:400] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:50.448762 15051 leader_election.cc:403] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.447938 14985 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.061908 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:50.463580 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 401631us
I20260709 13:04:50.452417 15051 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:50.470094 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.003s
W20260709 13:04:50.470688 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.004s	sys 0.003s
W20260709 13:04:50.473148 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.004s	sys 0.004s
W20260709 13:04:50.473721 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.005s	sys 0.004s
W20260709 13:04:50.476583 15159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.007s
W20260709 13:04:50.477172 15159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.007s
I20260709 13:04:50.312296 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2415) took 1038 ms. Trace:
W20260709 13:04:50.478708 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 422046us
I20260709 13:04:50.341059 15845 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:50.484122 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1301) took 1635 ms. Trace:
I20260709 13:04:50.485428 15160 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:50.326826 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:50.479017 15236 rpcz_store.cc:276] 0709 13:04:49.274044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.787209 (+513165us) service_pool.cc:224] Handling call
0709 13:04:49.787529 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.787856 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:49.892480 (+104624us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.304300 (+411820us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.311878 (+  7578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.311974 (+    96us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:50.312056 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.312127 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.312241 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129409,"prepare.queue_time_us":105529,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":35242,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":5048,"spinlock_wait_cycles":12538112,"thread_start_us":392517,"threads_started":1,"wal-append.queue_time_us":489197}
W20260709 13:04:50.512768 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.489517 15230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.002s	sys 0.018s
I20260709 13:04:50.555099 20444 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:50.564512 15230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.002s	sys 0.022s
W20260709 13:04:50.557435 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 252816us
W20260709 13:04:50.351989 20401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.021s	sys 0.052s
I20260709 13:04:50.318573 20465 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:50.460155 14985 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 13, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 13 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.591714 20335 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:50.615759 20659 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:04:50.624015 20461 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:04:50.625494 15235 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:50.625351 20461 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:50.625177 14985 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:50.633649 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:50.635242 20382 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:50.641059 20251 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:50.641809 20251 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:50.643911 20322 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.529465 15847 rpcz_store.cc:276] 0709 13:04:48.848193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.849685 (+  1492us) service_pool.cc:224] Handling call
0709 13:04:48.850118 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.850565 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.850606 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.765106 (+914500us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b1800404c20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a
0709 13:04:49.766156 (+  1050us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b2c000fb978. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:04:49.766691 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.242648 (+475957us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.242815 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.242877 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.483442 (+240565us) raft_consensus.cc:1685] finished
0709 13:04:50.483507 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.483865 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":519,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2451,"spinlock_wait_cycles":21476352,"thread_start_us":473879,"threads_started":1,"wal-append.queue_time_us":582244}
I20260709 13:04:50.636380 20382 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:50.665228 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 200167us
I20260709 13:04:50.664232 20136 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.650758 20487 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:50.675886 20481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.008s	sys 0.017s
I20260709 13:04:50.674177 20322 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:50.693553 20444 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:50.696748 20481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.009s	sys 0.018s
I20260709 13:04:50.706271 20429 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:04:50.708478 20466 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:50.709110 20466 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.709556 20466 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.655349 14985 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.714813 20456 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:50.715004 20335 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.716444 20444 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:50.720394 20335 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:50.773074 20136 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.777228 20437 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:50.778508 20382 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:50.782666 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81893.4us
I20260709 13:04:50.786222 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:50.786386 20429 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:50.787997 20465 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:50.789542 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 337439us
I20260709 13:04:50.791059 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:50.792254 20456 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:50.794932 20456 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.680488 20251 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.712251 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:50.792510 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:50.826169 20140 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:50.827085 20140 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:50.813602 20456 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.776468 14985 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 11, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.800055 20136 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:50.861548 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 69254us
W20260709 13:04:50.888394 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 215845us
I20260709 13:04:50.890048 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:50.894693 20251 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.902140 20322 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:50.902781 20481 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:50.903661 20481 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:50.904325 20481 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:50.888100 15156 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:50.901597 20646 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:50.912475 15156 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:50.927371 20663 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.217s	user 0.003s	sys 0.001s
W20260709 13:04:50.946980 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 418366us
I20260709 13:04:50.927979 20136 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.927821 14985 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 4 to 10, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 10 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:50.916774 20466 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.007093 20687 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.008458 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:50.993184 20140 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.030534 15156 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:51.003715 20444 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:51.014391 14985 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 12, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.039340 20532 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:51.039892 20532 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.039966 20520 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:51.040541 20520 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.041198 20532 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.042035 20520 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.043028 20532 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:51.044301 20522 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:04:51.046425 20520 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:51.050200 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.052476 20690 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:51.055944 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.061161 20646 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:51.061841 20646 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:51.063354 20646 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.065208 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1196) took 2065 ms. Trace:
W20260709 13:04:51.065496 20209 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.780s	user 0.017s	sys 0.000s
I20260709 13:04:51.066053 20209 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.067608 15844 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:51.068423 20444 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.070783 20444 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.065771 15156 rpcz_store.cc:276] 0709 13:04:48.999364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.000291 (+   927us) service_pool.cc:224] Handling call
0709 13:04:49.009195 (+  8904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.886272 (+877077us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540015eb20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:04:50.783331 (+897059us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540015eb20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f96206f 00007f0d3f9daf60 00007f0d3f9dc7de 00007f0d3f9e6b49 00007f0d3f9e7416 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62
0709 13:04:51.033488 (+250157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.065112 (+ 31624us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537497728}
I20260709 13:04:51.073257 20209 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.073918 20466 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:51.075152 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.044219 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 154329us
I20260709 13:04:51.077163 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:51.080050 20532 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:51.081276 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.081405 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.082198 15156 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 9.
I20260709 13:04:51.080583 20532 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.083148 20532 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.083320 20646 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.085572 14985 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index -1 to 5, term changed from 4 to 11, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to <none>, NON_VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) added. New cstate: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } } }
I20260709 13:04:51.087075 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.087471 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.087947 15155 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:51.087594 14985 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 10, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 10 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:04:51.089522 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.090042 14985 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 9 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.091085 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:51.094388 20692 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:51.097504 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.099087 15156 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:51.102296 20532 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:51.103384 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2848) took 2131 ms (client timeout 2350 ms). Trace:
W20260709 13:04:51.108287 15083 rpcz_store.cc:269] 0709 13:04:48.971576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.192559 (+220983us) service_pool.cc:224] Handling call
0709 13:04:51.102968 (+1910409us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400306e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:51.103188 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861378816}
I20260709 13:04:51.106262 20481 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.110461 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.111013 20528 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:51.111294 20481 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.111503 20528 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:51.112299 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.113137 15155 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:51.112766 20528 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.114358 20528 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:51.122082 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 326750us
W20260709 13:04:51.123564 20686 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.414s	user 0.001s	sys 0.001s
I20260709 13:04:51.123649 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1210) took 1234 ms. Trace:
I20260709 13:04:51.125459 15154 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:51.122411 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.128046 15159 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:51.124287 15160 rpcz_store.cc:276] 0709 13:04:49.889668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.890793 (+  1125us) service_pool.cc:224] Handling call
0709 13:04:51.123266 (+1232473us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:51.123546 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129840512}
I20260709 13:04:51.052397 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:51.131655 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79416.6us
I20260709 13:04:51.133131 20522 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:51.133960 20522 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.134542 20382 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:51.132784 15128 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.135293 20382 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.136724 20520 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:51.137931 20682 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.138691 20682 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.141001 20466 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:51.038744 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 335566us
W20260709 13:04:51.134166 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.141896 20466 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.148329 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:51.141440 20608 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.811s	user 0.015s	sys 0.002s
I20260709 13:04:51.147333 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:51.153337 20353 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.733s	user 0.004s	sys 0.006s
I20260709 13:04:51.159138 14985 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 6 to 12, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.158382 20608 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.166689 15848 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:51.158758 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.173285 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.114369 20532 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.187039 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2424) took 1464 ms. Trace:
I20260709 13:04:51.135316 20522 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.190074 15083 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:51.191670 15202 leader_election.cc:400] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:51.218931 15202 leader_election.cc:403] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:51.191923 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.192525 20364 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:51.194054 20459 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:51.195403 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:51.195583 20604 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.196729 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:51.199657 20651 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:51.200286 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:51.200482 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.201115 20509 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:51.201228 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:51.201367 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1161) took 1276 ms. Trace:
I20260709 13:04:51.201992 20655 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.202641 20360 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:51.200681 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2423) took 1478 ms. Trace:
I20260709 13:04:51.203686 15084 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:04:51.204913 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.205156 20653 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.206422 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.206957 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2437) took 1292 ms. Trace:
W20260709 13:04:51.209390 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 76770.7us
W20260709 13:04:51.209885 20677 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.355s	user 0.001s	sys 0.002s
I20260709 13:04:51.209681 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.211609 15088 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:51.212812 20683 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.173s	user 0.001s	sys 0.001s
I20260709 13:04:51.211545 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1370) took 1112 ms. Trace:
I20260709 13:04:51.234019 15087 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:51.213330 20682 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.229049 15154 rpcz_store.cc:276] 0709 13:04:49.914909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.010639 (+ 95730us) service_pool.cc:224] Handling call
0709 13:04:51.125064 (+1114425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.126044 (+   980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.206737 (+ 80693us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540033e350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:51.206904 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2561895680}
I20260709 13:04:51.215225 15232 rpcz_store.cc:276] 0709 13:04:49.722579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.882904 (+160325us) service_pool.cc:224] Handling call
0709 13:04:49.988591 (+105687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.014562 (+ 25971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.014596 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.015178 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.474749 (+459571us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:50.475631 (+   882us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.185177 (+709546us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b18003ba0a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:51.185431 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.185554 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.185601 (+    47us) raft_consensus.cc:1685] finished
0709 13:04:51.185731 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.186610 (+   879us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540005bc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:51.186815 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104709,"prepare.queue_time_us":62414,"prepare.run_cpu_time_us":1509,"prepare.run_wall_time_us":20353,"raft.queue_time_us":744043,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":268,"spinlock_wait_cycles":280649472,"thread_start_us":458929,"threads_started":1,"wal-append.queue_time_us":913193}
I20260709 13:04:51.162431 20353 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.231256 15079 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:51.240437 15085 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:04:51.241993 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 224600us
I20260709 13:04:51.218053 20136 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.242973 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.224701 15233 rpcz_store.cc:276] 0709 13:04:49.924759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.055090 (+130331us) service_pool.cc:224] Handling call
0709 13:04:50.055351 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.056030 (+   679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.056059 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.072382 (+ 16323us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.842646 (+770264us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.847480 (+  4834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.863591 (+ 16111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.199595 (+336004us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b54000c4ca0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f959dbe 00007f0d3f964930 00007f0d3f9648d8 00007f0d3f964858 00007f0d3f964813 00007f0d3f963a9c 00007f0d4584f626 00007f0d45849dd8 00007f0d3f9a07cb 00007f0d3f9a32c0
0709 13:04:51.200919 (+  1324us) raft_consensus.cc:1685] finished
0709 13:04:51.200952 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.201193 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":397,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":13508,"spinlock_wait_cycles":102521728,"thread_start_us":447852,"threads_started":1,"wal-append.queue_time_us":597153}
I20260709 13:04:51.214767 14985 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 12, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.244514 15231 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.221148 20461 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:04:51.249538 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58279us
I20260709 13:04:51.250392 15228 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.226790 15230 rpcz_store.cc:276] 0709 13:04:49.722351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.875215 (+152864us) service_pool.cc:224] Handling call
0709 13:04:49.875494 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.908682 (+ 33188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.908713 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.909421 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.683797 (+774376us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.701746 (+ 17949us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.200302 (+498556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.200410 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.200452 (+    42us) raft_consensus.cc:1685] finished
0709 13:04:51.200465 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.200619 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":479713,"prepare.queue_time_us":4980,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":31704,"raft.queue_time_us":513837,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":203,"thread_start_us":668116,"threads_started":1,"wal-append.queue_time_us":903221}
I20260709 13:04:51.255023 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.247649 20608 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.234284 15846 rpcz_store.cc:276] 0709 13:04:50.098748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.099669 (+   921us) service_pool.cc:224] Handling call
0709 13:04:50.219792 (+120123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.220066 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:50.316746 (+ 96680us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.962847 (+646101us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.126218 (+163371us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b18004c4d40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:51.210706 (+ 84488us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b18004c4d40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:51.210947 (+   241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.211011 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:51.211085 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.211204 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.211369 (+   165us) inbound_call.cc:177] Queueing success response
0709 13:04:51.211975 (+   606us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
Metrics: {"mutex_wait_us":9773,"prepare.queue_time_us":20290,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":6205,"raft.queue_time_us":234038,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":29829,"spinlock_wait_cycles":48716800,"thread_start_us":384035,"threads_started":1,"wal-append.queue_time_us":686312}
W20260709 13:04:51.269459 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.310272 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.313588 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.314091 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.315181 20314 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.315862 20314 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.259609 20532 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.246644 20653 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.316967 20651 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.317296 20532 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.318835 20653 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.313652 15845 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:51.319453 20532 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.319901 20651 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.320466 20314 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.321064 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:51.321337 20701 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.276s	user 0.002s	sys 0.000s
I20260709 13:04:51.322835 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.324066 15229 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:51.326133 20664 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:04:51.327132 20532 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:51.327668 20532 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.328637 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:51.329368 15845 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:51.329258 20584 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:51.329883 15842 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:51.331220 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.331832 15842 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:51.332898 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.333232 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.334009 15843 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:51.328677 20532 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.335860 15841 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:51.336369 20532 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.340169 15845 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:04:51.340072 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 84008.2us
I20260709 13:04:51.344990 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.347775 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:51.309278 14985 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 11, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 11 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:04:51.351145 20276 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.017s	user 0.006s	sys 0.013s
I20260709 13:04:51.259423 20481 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.256013 20456 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.369484 15233 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.349131 20382 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.256294 20459 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.252627 20509 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.254691 20655 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:51.380060 20695 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.198s	user 0.000s	sys 0.002s
I20260709 13:04:51.380609 20680 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:04:51.381834 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.382530 20423 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.259757 20604 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.363688 20386 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:51.384124 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59971.9us
I20260709 13:04:51.361084 20251 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.380713 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2425) took 1657 ms. Trace:
I20260709 13:04:51.363317 20584 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:51.388564 20602 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:51.388803 20688 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.389079 20612 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.455144 20689 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:51.455814 20612 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.389837 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:51.456502 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.457451 15088 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:51.390763 20459 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.458256 20655 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.428958 20276 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:04:51.459847 20612 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.364207 20466 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.365131 20646 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:51.461298 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:51.466929 20509 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.467525 20690 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:51.468146 20690 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.469831 20646 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.470635 20690 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.472462 15840 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:51.474215 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:04:51.388031 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.008s	sys 0.008s
I20260709 13:04:51.394484 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:51.476181 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.008s	sys 0.008s
I20260709 13:04:51.399853 15083 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:51.394222 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.400525 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.399576 20045 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:04:51.480296 20509 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:51.480782 20509 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.481227 20509 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:51.481314 15161 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:51.402015 20653 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:04:51.484721 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.455747 20688 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.485571 15234 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:51.487069 20688 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.406216 20604 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.408505 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.379324 20481 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.409914 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1380) took 1132 ms. Trace:
I20260709 13:04:51.489862 15848 rpcz_store.cc:276] 0709 13:04:50.277393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.278280 (+   887us) service_pool.cc:224] Handling call
0709 13:04:50.278638 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.174554 (+895916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.409626 (+235072us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:51.409863 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2041571968}
I20260709 13:04:51.410279 20378 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.491215 20378 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.411306 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.412264 20289 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.413527 15082 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:51.414312 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.493260 20289 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.494390 20276 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:51.494840 20276 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.495652 20289 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.495791 20276 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.497080 20383 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.497329 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:51.497334 20378 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.414921 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.498508 20289 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.498874 15085 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:51.415958 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.499397 20289 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:51.417114 20188 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.500312 20289 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:51.500765 20289 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:51.418675 20608 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.501400 20276 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:51.501638 20608 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.501830 20289 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.503577 20289 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:51.504261 20608 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.418987 20209 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.505123 20209 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.419966 15236 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 13:04:51.422451 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.506587 15230 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:51.507306 20209 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.500543 20188 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.386850 20136 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:51.378940 20609 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.942s	user 0.009s	sys 0.008s
I20260709 13:04:51.424093 15228 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:04:51.357782 20360 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.432345 14985 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.511593 20188 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.435320 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.451802 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:51.451933 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.452004 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.514042 15154 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:04:51.452744 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.453279 20602 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:51.453608 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:51.476476 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 82054.9us
I20260709 13:04:51.497645 20383 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:51.415083 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.499796 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.377027 20651 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:04:51.515048 15235 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:51.538802 15231 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:51.535431 15233 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:51.543385 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:51.539758 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50785.2us
I20260709 13:04:51.549680 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:51.553581 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.558547 15846 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.559310 14985 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 16, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.428361 15237 rpcz_store.cc:276] 0709 13:04:49.722801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.889142 (+166341us) service_pool.cc:224] Handling call
0709 13:04:49.889478 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.921933 (+ 32455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.921980 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.922504 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.071902 (+149398us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b180047bf60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:04:50.476745 (+404843us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:50.476823 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.477011 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.208908 (+731897us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:51.380269 (+171361us) raft_consensus.cc:1685] finished
0709 13:04:51.380299 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.380479 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32052,"raft.queue_time_us":50863,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":41379,"spinlock_wait_cycles":54080640,"thread_start_us":404218,"threads_started":1,"wal-append.queue_time_us":813573}
I20260709 13:04:51.562577 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.565742 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.566334 15156 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.567582 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:51.567065 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1215) took 1096 ms. Trace:
W20260709 13:04:51.576275 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.539152 20584 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.577095 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:51.578382 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62848.9us
I20260709 13:04:51.582139 15161 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:51.577752 15234 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:51.584507 20681 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:51.570833 20609 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.589843 20456 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.589524 20609 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:51.592955 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.546193 20364 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:51.595218 20609 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.595405 20314 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:51.592259 20251 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.596195 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96520.7us
I20260709 13:04:51.596879 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.596479 20584 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:51.597900 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.600971 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 200518us
I20260709 13:04:51.428244 20423 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.555696 20459 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:51.544509 20653 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:51.604135 20378 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:51.610287 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.611706 20360 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.611876 15161 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:51.613554 20383 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.622012 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.622594 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.624053 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.629381 20251 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:51.632481 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 180733us
W20260709 13:04:51.680361 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 248528us
W20260709 13:04:51.683768 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58915.1us
W20260709 13:04:51.686007 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 91328.1us
I20260709 13:04:51.573076 15157 rpcz_store.cc:276] 0709 13:04:50.471408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.482493 (+ 11085us) service_pool.cc:224] Handling call
0709 13:04:50.483114 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.484128 (+  1014us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:50.689426 (+205298us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.484258 (+794832us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:51.560604 (+ 76346us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.560685 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:51.560737 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.560803 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.567006 (+  6203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71560,"prepare.queue_time_us":320391,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":69900,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":47000,"spinlock_wait_cycles":43790208,"thread_start_us":975673,"threads_started":2,"wal-append.queue_time_us":676844}
I20260709 13:04:51.601631 20522 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.593369 20353 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:51.693521 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 140442us
I20260709 13:04:51.686435 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.701969 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.608309 20681 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:51.705843 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:51.708243 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.680341 20481 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.712253 15237 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:04:51.715607 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 170260us
I20260709 13:04:51.714138 14985 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.689882 20383 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.689385 20314 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:51.724157 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.725097 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:51.717592 20651 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:51.729866 14985 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 9, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.731542 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:51.722760 20382 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.720448 20528 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:51.755515 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 233866us
I20260709 13:04:51.724438 20188 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.759454 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.764605 15156 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
W20260709 13:04:51.764777 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66037us
I20260709 13:04:51.733016 20655 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.748116 20651 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:51.749010 20360 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:51.726949 20383 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.786376 20383 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.759630 20289 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:51.795924 20289 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.796355 20289 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.780746 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:51.802759 20646 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:51.803321 20646 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.803385 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:51.803874 20646 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:51.781006 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 101951us
I20260709 13:04:51.805295 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.805712 20314 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.806595 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.807617 20580 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.808233 20580 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:51.809187 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.810024 15845 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:51.810920 20688 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.811710 20688 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.814512 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.814692 20382 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.815668 20653 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.726202 20481 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:04:51.817471 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.783152 20188 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.819013 20602 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.820509 20188 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.814574 20688 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:51.783196 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.822777 20423 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.822445 14985 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.824440 15160 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:51.737039 20681 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.839444 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.849084 20532 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:51.850899 20532 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:51.851401 20532 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:51.854131 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.854308 20188 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:51.856045 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.857307 14985 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 7 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.861377 20680 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:51.862185 20680 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:04:51.862512 20712 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:04:51.781441 20651 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:51.863386 20680 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.869359 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.870374 15154 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:51.871330 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.871949 15159 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:51.872506 20522 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.872874 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.873231 20653 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:51.873109 20522 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.873739 15162 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:51.874154 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:51.836794 20688 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:51.874631 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.875361 15163 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:51.875012 20688 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.875876 20522 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.851220 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.859671 14985 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 9, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 9 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.833863 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:51.879743 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 117460us
W20260709 13:04:51.883136 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49406.6us
W20260709 13:04:51.889654 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.891173 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:51.893739 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:51.888345 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.758802 20353 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:04:51.775297 20682 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.127s
I20260709 13:04:51.856441 20188 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.900933 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:51.901178 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.918651 15080 raft_consensus.cc:2410] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:51.900630 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.820749 20602 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:51.922598 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66654.7us
I20260709 13:04:51.924176 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:51.927940 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43743.8us
I20260709 13:04:51.936802 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:51.903151 20653 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.938370 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:51.837293 20466 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:04:51.874182 20681 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.943254 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:51.920823 20353 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:51.947288 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 96195us
I20260709 13:04:51.948168 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:51.951126 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:51.942663 14985 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 11, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 11 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.952929 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:51.937956 20522 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:51.957491 20522 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.960750 20522 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:51.896764 20360 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.943302 20690 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.958477 14985 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 11, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 11 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:04:51.943436 20437 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:51.949743 20682 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:51.983831 20682 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:51.852787 20532 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.967721 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.967931 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:51.926394 20655 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:51.969239 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:51.969440 20045 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:51.969928 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:51.970666 20382 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:51.971285 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 131961us
I20260709 13:04:51.971822 20379 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:51.948412 20188 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:51.974521 20360 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:51.975548 20580 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:51.966564 20353 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:04:51.981168 20584 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.963038 20688 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:51.958751 20509 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.983385 20437 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.994472 15846 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:51.993191 20045 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.993534 15159 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:51.997627 14985 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.992610 15079 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:52.000398 15160 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.995262 20251 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:51.995581 20379 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.960183 20289 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:51.989781 20609 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:52.024998 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68499.1us
W20260709 13:04:52.030470 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 79773.4us
I20260709 13:04:52.024434 20604 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:52.026790 20437 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.027033 20689 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:52.029830 20045 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.044929 15229 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:52.033605 20379 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:52.053574 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100732us
I20260709 13:04:52.044072 20604 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:52.058604 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.059298 15079 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:52.060025 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 121050us
W20260709 13:04:52.061827 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 116346us
I20260709 13:04:52.040233 20382 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.066491 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.066443 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.067337 15162 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:52.071172 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.073773 15230 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:52.074800 20723 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.075434 20723 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.079905 20681 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.086570 20615 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.088429 20717 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.060644 20604 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.090590 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:52.094148 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.090004 20251 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.039924 20609 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.095386 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.096069 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.091935 20289 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.103812 15082 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:52.105041 15085 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:52.108560 15088 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:52.111474 20720 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:52.111997 20720 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.112378 20720 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.112429 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.113646 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.114459 15080 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:52.116209 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.118003 15155 pending_rounds.cc:85] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:52.118477 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.068327 20522 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.071971 20382 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.091814 20651 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.118361 15155 pending_rounds.cc:107] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:52.128620 15155 raft_consensus.cc:2889] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:52.131606 20723 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.136282 20382 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.137157 20437 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.138294 20609 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.139345 20509 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.140267 20437 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:52.142481 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 52028.9us
I20260709 13:04:52.143361 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.127007 15079 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:52.150597 20722 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.105988 20717 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.155066 20717 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.155818 20655 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:52.107784 20251 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.157210 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.157629 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.157909 15083 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:52.158488 15084 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:52.156829 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.158897 20251 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.161367 15083 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.160537 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.161290 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.162240 20682 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.163352 20682 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.166458 20456 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:52.166898 20456 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:52.167721 20456 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.168272 15086 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.169032 15084 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.170018 20509 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:52.170845 20481 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:52.171247 20481 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:52.172084 20481 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.174093 20580 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:52.174376 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.176252 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.098694 20615 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:04:52.177573 15083 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:52.178293 15156 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:52.187683 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:52.188190 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.189141 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.189704 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.188661 15237 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:52.120107 20314 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.192153 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.191926 20604 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.192787 15081 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:52.193234 15232 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:52.194073 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.195178 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.195942 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.196502 15163 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.196924 20723 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.197605 20723 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.199066 20646 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.200139 20723 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:52.200515 15082 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:52.201160 20604 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.203182 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:52.203850 20481 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.204962 20609 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.205516 20609 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.206379 20651 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:52.206933 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46896.8us
I20260709 13:04:52.207623 20609 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.207687 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.208355 20609 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.208984 20456 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.209172 20602 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:52.210392 20602 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.213407 20612 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:52.213835 20612 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:52.214066 20615 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:52.214567 20615 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:52.214740 20612 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.215610 20615 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.216282 20612 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.217584 20615 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.217775 20522 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.217764 20557 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.218976 20557 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.224709 20557 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.225507 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.226900 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.228319 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.228430 20722 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.229220 20722 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:52.231230 15079 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:52.231880 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.232183 15846 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.232371 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.233048 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.233129 15840 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:52.233367 20532 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.234299 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:52.234766 15230 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.236006 20466 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:52.236160 15157 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:52.236469 20466 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.237268 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.238588 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.239202 15158 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:52.241181 15159 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.242899 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.243441 15155 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:52.244566 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.249922 15155 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:52.245335 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.251114 15159 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:52.258157 20602 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.262228 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.263108 20690 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:52.263621 20386 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.263979 20690 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:52.264353 20690 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:52.264147 20386 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.263298 20687 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:52.266223 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.269248 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.269618 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.270367 15084 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:04:52.265585 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47980.6us
I20260709 13:04:52.270439 15086 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:52.272349 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.281276 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:04:52.251566 20720 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.307772 20045 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:52.309514 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 65017.2us
I20260709 13:04:52.317762 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:52.317725 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51601.6us
I20260709 13:04:52.319868 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.320887 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.265331 20466 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.317734 20604 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.333407 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.334676 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:52.317788 20522 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:52.336766 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:52.330922 20609 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.303407 20689 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.331194 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.331913 20687 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:52.348610 15841 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:52.332052 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.339154 15083 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:52.332602 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.341068 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.332017 15230 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:52.349795 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.356696 15086 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:52.356906 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.350728 15079 raft_consensus.cc:2376] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14. Re-sending same reply.
I20260709 13:04:52.357604 15084 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:52.304289 20364 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:52.365134 20557 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.370586 20557 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.311007 20314 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:52.352782 20520 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:52.313427 20651 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:04:52.378721 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.385982 15163 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.386374 15081 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:52.348824 20687 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:52.404717 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.360706 20580 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:52.407548 15232 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:04:52.423048 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68100.8us
I20260709 13:04:52.345389 20136 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:04:52.360162 20378 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:52.385246 20716 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.268s	user 0.004s	sys 0.000s
I20260709 13:04:52.439831 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:52.441375 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 121586us
I20260709 13:04:52.443957 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.445459 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.446763 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.449420 15082 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:52.449452 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.446208 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.411405 20520 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.469779 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.472289 15237 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:52.458016 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 137245us
I20260709 13:04:52.474440 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.265324 20690 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.477530 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.479264 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.480129 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.482383 15079 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:52.485196 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.487833 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:52.491048 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.489558 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.440369 20602 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:52.432631 20522 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:52.490206 15082 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:04:52.495929 20723 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:52.521799 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:52.522624 15810 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.481719 20520 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:52.496276 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.498095 15230 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.509598 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:52.538410 15845 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.511186 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.495003 15841 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:52.516463 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:52.486114 15086 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:04:52.529646 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.540457 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.531052 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.533468 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.488863 20655 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.486881 20378 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:52.546759 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.458057 20276 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:04:52.504010 20689 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:52.572510 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47702.9us
I20260709 13:04:52.533701 20722 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:52.582005 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.533432 20723 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.586902 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.534003 20437 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.584314 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:52.592674 15086 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.542285 20584 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:52.547367 20209 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:52.591192 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.549322 20251 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:52.596046 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.488551 20379 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.547780 20557 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.591140 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:52.600308 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.600606 20289 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.549479 20717 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.567488 20738 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:52.621011 15841 raft_consensus.cc:2368] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:04:52.622340 20386 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.598979 20723 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:52.625039 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.626540 15201 leader_election.cc:400] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:04:52.630169 15201 leader_election.cc:403] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:52.631023 20509 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:52.631101 20727 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:52.589918 20646 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.645401 15848 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:52.645725 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.646524 15230 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:52.606470 20209 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:52.491132 20735 consensus_queue.cc:1175] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 12 status { last_received { term: 12 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 12 index: 9 } }
I20260709 13:04:52.614647 20682 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.617683 20720 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:52.615320 20045 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.661792 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.667294 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:52.667816 15233 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:52.617364 20738 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:52.662087 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:52.660910 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 304192us
I20260709 13:04:52.664026 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.672992 15232 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.662528 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.673943 15235 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:52.682276 20731 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:04:52.685168 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.685544 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.685062 20522 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.686050 15847 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:52.686677 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.687119 15847 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:52.687259 15841 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:52.659284 20209 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:52.689881 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:52.678843 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.699095 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.700205 15230 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:04:52.707963 15845 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:52.659958 20736 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:52.626940 20251 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.721297 15201 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:52.717779 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.724176 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.727540 15841 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:04:52.649672 20717 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.732177 20580 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.729228 20045 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:52.693852 20378 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.736317 20188 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:52.737843 20383 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.738838 20364 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.747550 15843 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:52.732605 20314 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:52.746477 20423 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:52.753243 20737 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:04:52.754571 20744 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:52.756006 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.757865 20276 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:52.760232 20532 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.004s
I20260709 13:04:52.680410 20682 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.764626 20188 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:52.743506 20604 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.743470 20509 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:04:52.748564 20045 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.749040 20580 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:52.814555 20481 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.001s
I20260709 13:04:52.815081 20580 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:52.754750 20251 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.746893 20736 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.779000 20314 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:52.817215 20314 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.818285 20736 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.791157 20682 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.784113 20364 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.790314 20188 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.792137 15229 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.826049 20423 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:52.750240 20466 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.795396 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.826679 20423 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.798715 20045 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:52.828465 20466 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.807179 20532 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:52.833292 20532 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:52.834146 20532 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.835407 20532 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.781152 20276 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.837061 20276 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.840462 15232 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:04:52.829329 15080 raft_consensus.cc:2368] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
W20260709 13:04:52.803303 15085 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.034s	sys 0.064s
I20260709 13:04:52.841336 15230 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:52.841552 15085 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.034s	sys 0.064s
I20260709 13:04:52.809157 20379 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:52.850149 15237 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:52.835146 20045 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.853411 15235 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.853497 20651 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.860585 20651 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:52.863643 15231 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.864108 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.865008 15230 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.855315 15234 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:52.853705 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.867368 15233 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.874667 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.876329 15086 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.878046 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:52.761112 20738 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.882241 15847 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:52.882661 20738 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.883665 20723 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:52.885406 20732 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:04:52.886914 20743 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:04:52.886264 20723 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:52.887717 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:52.889990 20687 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.891259 20687 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:52.892000 20717 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.896026 15088 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:52.848304 20509 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:52.897332 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:52.897246 20509 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:52.898522 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.899412 20136 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:52.899802 20136 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.899936 20509 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:52.900151 20136 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.905098 20723 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:52.881084 20285 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:52.918277 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.918906 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:52.926728 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.927392 15235 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:52.936040 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.933480 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.931799 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.941187 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:52.931529 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:52.934087 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 216397us
I20260709 13:04:52.943884 15084 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:52.944152 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.941255 15079 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:52.945582 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.945721 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.946341 15229 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:52.946429 15232 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 10.
I20260709 13:04:52.952907 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:52.955503 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.956439 15236 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:52.960067 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.913791 20651 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.963950 15202 leader_election.cc:400] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:52.971555 15202 leader_election.cc:403] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:52.964641 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.972177 15154 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:52.972678 15845 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:52.954818 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:52.966049 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.973822 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.969218 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:52.972462 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:52.976344 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 297841us
I20260709 13:04:52.977913 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:52.978920 15231 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
W20260709 13:04:52.982126 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.985456 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.986230 15159 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:52.986011 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.986805 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.925671 20723 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:52.942509 20717 raft_consensus.cc:2764] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:53.002023 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:53.003002 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:53.003494 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:53.003654 15848 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:52.944808 20437 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:52.934437 20744 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:52.991993 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.006188 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.988900 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:52.993988 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:52.994978 15233 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:04:53.009558 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45989us
I20260709 13:04:53.010869 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.012357 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:53.013321 15236 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.015300 20744 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.003657 20690 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:53.017716 20744 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:52.967730 20466 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:53.019317 20466 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:53.020288 20466 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.021310 20251 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:53.021799 20251 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.022193 20251 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.023032 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.023298 20276 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.023996 20045 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:53.025193 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:53.025780 20744 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:53.026240 20744 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:52.945266 20285 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:53.026491 20364 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.027277 20285 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.028193 20655 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.028975 15228 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:52.999747 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:53.020599 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.032413 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.033068 20386 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:53.021358 20721 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:04:52.970594 20604 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:53.067714 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.067800 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.068600 15155 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:53.035233 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:53.025213 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:52.987434 15844 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:52.989390 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:53.040318 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.071136 20522 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:53.050479 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:53.073380 15235 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:52.970397 20612 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.053303 15230 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:04:52.989033 20682 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.927886 20136 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.033604 20382 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:53.071075 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 108595us
I20260709 13:04:53.051481 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:52.987829 20209 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.078310 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:53.132190 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.081878 15158 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:53.161348 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:53.132968 15237 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:04:53.138389 15228 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:53.139536 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.049610 20744 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:53.039916 20188 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:53.049499 20386 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.037037 20752 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:53.142308 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 178471us
I20260709 13:04:53.148198 15156 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.068437 15161 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:53.199374 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:53.156589 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:53.157182 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:53.200560 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:53.201722 15235 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:53.163163 15163 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.159950 15157 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:04:53.166090 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 195102us
I20260709 13:04:53.140450 20680 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:53.213469 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 219597us
I20260709 13:04:53.222178 15846 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:53.147002 20466 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.079027 20520 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.234045 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.233544 15052 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.161646 20721 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:53.081339 20604 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.134431 20612 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.197585 20386 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:53.256183 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52895.3us
I20260709 13:04:53.221843 20680 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.245308 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.200111 20682 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:53.211135 20522 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:53.264252 15156 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:04:53.265184 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 254518us
I20260709 13:04:53.222335 20188 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:53.267534 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 235188us
W20260709 13:04:53.269964 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 69390.8us
I20260709 13:04:53.271262 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.278451 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:53.278851 15163 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:53.276542 15848 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:04:53.274977 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.261384 20735 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.266201 20680 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.267424 20682 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:53.224040 20752 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.273761 20522 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:53.349534 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61558.5us
I20260709 13:04:53.277731 20276 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.273573 20750 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:53.277041 20690 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:53.353390 15126 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:53.377614 20749 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:04:53.378108 20750 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.370908 20717 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:04:53.382211 20717 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:53.382933 20717 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:53.385825 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:53.387316 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:53.388458 20746 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:53.384106 20717 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:53.397962 20765 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:53.389135 20756 raft_consensus.cc:2764] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:53.399341 20756 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:53.399834 20756 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.400207 20756 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:53.400107 20188 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:53.403827 20748 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
I20260709 13:04:53.407187 20750 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.233191 20723 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.284525 20745 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:53.289537 20736 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.414712 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 173422us
I20260709 13:04:53.280141 20437 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:04:53.354569 20382 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.356526 20522 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.296430 20609 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:53.477404 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 90185.4us
I20260709 13:04:53.362003 20761 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:53.460376 20748 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
W20260709 13:04:53.478734 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 164791us
W20260709 13:04:53.493939 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 219395us
I20260709 13:04:53.367028 20753 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:04:53.371824 20251 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.351490 20767 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.190s
I20260709 13:04:53.504627 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.506065 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.523785 15163 raft_consensus.cc:2368] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:53.523864 15156 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:53.510119 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2239) took 1338 ms. Trace:
I20260709 13:04:53.510656 20717 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:53.517912 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.489693 20456 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.572144 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.573237 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.476771 20736 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.583890 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.584923 15156 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:53.587306 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.524595 15085 rpcz_store.cc:276] 0709 13:04:52.155199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.155324 (+   125us) service_pool.cc:224] Handling call
0709 13:04:52.155641 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.155948 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:52.842028 (+686080us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.487501 (+645473us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.487579 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:53.487644 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.490278 (+  2634us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.493941 (+  3663us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16390,"prepare.queue_time_us":260155,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":92192,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2562,"thread_start_us":1313786,"threads_started":3,"wal-append.queue_time_us":355833}
I20260709 13:04:53.512120 20609 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:53.615061 15236 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.002s
I20260709 13:04:53.622761 15236 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:53.559737 20717 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:53.627431 20461 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.628197 20687 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.000s
I20260709 13:04:53.628393 20461 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.580772 20136 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:53.601972 20759 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 13:04:53.594187 20209 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:53.642349 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.610682 20251 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.619617 20466 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:53.625671 20655 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:53.609455 20520 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:53.609153 20604 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.652765 15156 raft_consensus.cc:2368] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
W20260709 13:04:53.633392 20532 consensus_meta.cc:287] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.797s	user 0.007s	sys 0.004s
I20260709 13:04:53.595909 20763 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:53.617136 20612 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:53.595716 20774 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:53.488552 20761 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.606804 20736 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.512938 20682 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.601758 20752 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:53.591707 20767 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:53.665660 20461 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.595762 20781 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:04:53.670588 20209 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.723474 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 241614us
I20260709 13:04:53.597124 20771 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.168s
I20260709 13:04:53.655350 20466 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:53.725463 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:53.681438 20382 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.701862 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 441241us
I20260709 13:04:53.662530 20793 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:53.735328 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.676990 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:53.738200 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:53.739126 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:53.741714 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.668993 20386 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.679636 20763 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:53.752444 15842 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:53.684340 20774 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:53.752700 15085 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:53.754220 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:53.739475 20423 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.912s	user 0.007s	sys 0.008s
I20260709 13:04:53.754518 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:53.770358 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:53.735395 20612 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:53.753017 15229 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:53.751325 15841 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:53.744175 15230 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.007s	sys 0.018s
W20260709 13:04:53.778896 15230 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.007s	sys 0.018s
I20260709 13:04:53.769668 20774 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:53.791159 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.757230 20682 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:53.765235 20763 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.769742 20786 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:53.806967 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65432.2us
I20260709 13:04:53.816756 20779 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.817656 20779 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:53.819021 20738 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.936s	user 0.010s	sys 0.000s
I20260709 13:04:53.738773 20609 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.819576 20787 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:53.820147 20787 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:53.821755 20782 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:04:53.825747 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.826471 15157 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:53.828832 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:53.830322 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:53.830322 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.777381 20612 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:53.818135 20764 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.468s	user 0.002s	sys 0.000s
I20260709 13:04:53.837419 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:53.831943 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.726545 20717 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.840798 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.752916 20209 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.841810 15160 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.843755 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.846510 15085 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:04:53.846642 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:53.849435 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.874838 15842 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.762938 20382 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.847364 15154 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:53.885001 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:53.806062 20481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.010s	sys 0.014s
I20260709 13:04:53.828109 20767 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:53.780432 20423 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.820241 20779 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:53.822796 20787 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:53.805977 20721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.013s	sys 0.016s
I20260709 13:04:53.870682 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:53.920352 15839 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.858536 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.921485 15155 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:04:53.890182 20481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.010s	sys 0.015s
I20260709 13:04:53.889688 15203 leader_election.cc:400] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
W20260709 13:04:53.925707 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50236.2us
I20260709 13:04:53.897535 20780 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.927395 20780 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:53.929706 20780 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:53.841843 20738 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.930387 20738 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:53.904441 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 60760.5us
I20260709 13:04:53.905433 15160 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:53.933866 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.840984 20736 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:53.888707 20788 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:53.862253 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2443) took 1010 ms. Trace:
I20260709 13:04:53.845345 20789 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:53.934072 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:53.921948 20721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.013s	sys 0.017s
W20260709 13:04:53.937762 20755 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.773s	user 0.003s	sys 0.017s
I20260709 13:04:53.941908 20364 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.980520 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54138.9us
I20260709 13:04:53.982553 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.942879 20776 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.762794 20793 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.985376 15154 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:53.869031 20794 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:04:53.947111 15237 rpcz_store.cc:276] 0709 13:04:52.852149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.867493 (+ 15344us) service_pool.cc:224] Handling call
0709 13:04:52.867875 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.162398 (+294523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.861845 (+699447us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000acdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:53.862168 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536738560}
W20260709 13:04:53.953631 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 121755us
I20260709 13:04:53.936573 15086 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:53.949224 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:54.003082 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:53.911628 20386 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:53.829429 20680 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.810003 20466 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.974560 15839 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:53.977458 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2882) took 1048 ms. Trace:
I20260709 13:04:53.983749 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.960314 20678 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.012182 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:54.014554 15845 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:54.009500 15203 leader_election.cc:400] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:04:54.018316 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:53.947045 20788 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.043510 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:53.961167 20789 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.046510 20789 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:54.046842 20785 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:54.047570 20785 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.984920 20690 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.051849 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:54.008034 15081 rpcz_store.cc:276] 0709 13:04:52.929492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.943207 (+ 13715us) service_pool.cc:224] Handling call
0709 13:04:52.975854 (+ 32647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.015263 (+ 39409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.015300 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.015890 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.070219 (+ 54329us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:53.070268 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.070346 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.817390 (+747044us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:53.961532 (+144142us) raft_consensus.cc:1685] finished
0709 13:04:53.961563 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.977377 (+ 15814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29399,"raft.queue_time_us":40324,"spinlock_wait_cycles":214355584,"thread_start_us":53874,"threads_started":1,"wal-append.queue_time_us":219646}
I20260709 13:04:53.865036 20786 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.979807 20780 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.066422 20761 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.065665 20532 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:54.060683 15154 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:54.000216 20735 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:53.970870 20721 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:54.049827 20520 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.015247 20781 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:04:54.079293 15842 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:54.006017 20804 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:54.082337 20466 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:54.053437 20788 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:54.096009 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77796.8us
I20260709 13:04:54.141109 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:53.970320 20736 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:53.962432 20767 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:54.005432 20285 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:54.102991 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 139473us
I20260709 13:04:54.006740 20364 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.082345 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:53.961702 20612 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:54.097405 20580 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.009s	sys 0.029s
I20260709 13:04:54.080204 20762 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:54.044409 20678 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.144738 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:54.194700 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:04:54.152454 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 149482us
I20260709 13:04:54.105221 20779 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:54.100642 20689 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:54.196789 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:54.074290 20690 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:54.162190 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 213047us
I20260709 13:04:54.201237 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:54.201967 20690 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.202461 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:54.203246 15080 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:54.199072 20285 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.204057 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.195632 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:54.080648 20780 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.204648 15080 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:54.205205 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.205699 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.206141 15080 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:04:54.206405 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.206481 15080 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 11
I20260709 13:04:54.207275 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.134979 20775 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:54.206593 20780 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:54.199043 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54429.9us
I20260709 13:04:54.207425 15080 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:04:54.208293 15080 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:54.209194 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:54.209494 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.209748 15232 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:54.206820 15085 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:54.208081 15088 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:54.208724 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:54.140451 20761 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.139719 20812 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:54.215673 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.216588 15082 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 11: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:54.137032 20806 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:54.144917 20795 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:54.193150 20580 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.009s	sys 0.029s
W20260709 13:04:54.201622 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 372351us
I20260709 13:04:54.221868 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:54.222121 20773 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:54.217947 15228 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:54.223392 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.107257 20520 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.224329 15237 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:54.229548 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.225392 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1194) took 1089 ms. Trace:
I20260709 13:04:54.226599 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:54.230158 15231 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:54.230645 15230 rpcz_store.cc:276] 0709 13:04:53.136448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.137949 (+  1501us) service_pool.cc:224] Handling call
0709 13:04:53.138321 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.138570 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:53.779290 (+640720us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.206998 (+427708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.224526 (+ 17528us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:54.224615 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.225115 (+   500us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.225328 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":516232,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":1034914,"threads_started":2}
I20260709 13:04:54.107656 20735 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.211660 20745 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.223344 20136 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.244283 20830 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:54.144528 20804 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.217442 20780 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.258370 15841 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:54.219916 20812 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:54.273989 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:04:54.270576 20522 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.001s
I20260709 13:04:54.304148 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:54.308328 20690 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:54.309463 20690 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:54.309854 20690 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:54.310791 20690 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.312120 20690 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:54.312403 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.313047 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.313699 15232 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:04:54.313769 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:54.315187 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:54.316344 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:54.324640 20690 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:54.346199 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 143924us
W20260709 13:04:54.362802 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 168170us
I20260709 13:04:54.373150 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:54.374019 15082 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:54.275141 20382 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.215664 20689 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.347757 20690 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:54.384001 15203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.546s (SENT) [suppressed 19 similar messages]
I20260709 13:04:54.297185 20209 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.297253 20812 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:54.395351 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.350319 20761 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:54.274760 20136 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.518038 20678 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:54.518981 20136 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.519695 20832 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:54.519009 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:54.520931 20804 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.522925 15154 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:54.387583 20687 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:54.406322 20690 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.536934 20690 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:54.537076 20824 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:04:54.537572 20807 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 13:04:54.538164 20824 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.538683 20825 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:54.539196 20825 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.538816 20824 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.539554 20825 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:54.538112 20817 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.552410 20817 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:54.545650 20823 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:04:54.546701 20811 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 13:04:54.547340 20824 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:54.547756 20821 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.549330 20809 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:54.550073 20842 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:54.550515 20682 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.751s	user 0.008s	sys 0.000s
W20260709 13:04:54.551151 20816 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.325s	user 0.000s	sys 0.000s
I20260709 13:04:54.532269 20209 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:54.360368 20522 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.396179 20752 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.560163 20793 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.562263 20837 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:54.566232 20758 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.175s	user 0.002s	sys 0.000s
I20260709 13:04:54.567907 20786 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.565838 20690 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.409129 20285 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.300866 20780 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.523066 20382 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:54.261200 20721 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:54.597882 20744 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.387s	user 0.014s	sys 0.000s
I20260709 13:04:54.532617 20761 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.554937 20785 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:54.608481 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 412911us
I20260709 13:04:54.574229 20557 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:54.611677 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 283374us
I20260709 13:04:54.572515 20687 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:54.607795 20756 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.204s	user 0.014s	sys 0.000s
I20260709 13:04:54.609702 20842 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.603889 20809 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:54.768512 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 88362.9us
I20260709 13:04:54.776429 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:54.777981 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:54.816735 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 500416us
I20260709 13:04:54.775082 20557 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.825521 20735 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:54.826707 20815 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
I20260709 13:04:54.827196 20557 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.827664 20780 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.830494 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:54.776222 20842 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:54.775471 20736 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.768236 20382 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:54.834440 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2898) took 1324 ms. Trace:
I20260709 13:04:54.868939 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2901) took 1125 ms. Trace:
I20260709 13:04:54.804107 20809 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:54.765422 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 381429us
W20260709 13:04:54.878782 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48383.8us
I20260709 13:04:54.275146 20745 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.612864 20752 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.675490 20744 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.899536 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:54.597263 20837 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.774878 20682 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:04:54.868484 15163 rpcz_store.cc:276] 0709 13:04:53.509815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.524504 (+ 14689us) service_pool.cc:224] Handling call
0709 13:04:53.529812 (+  5308us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:53.624811 (+ 94999us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:53.806116 (+181305us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.260650 (+454534us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:54.300185 (+ 39535us) log.cc:844] Serialized 78 byte log entry
0709 13:04:54.300393 (+   208us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:54.301152 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.301294 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.834201 (+532907us) raft_consensus.cc:1685] finished
0709 13:04:54.834223 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.834383 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51717,"prepare.queue_time_us":357214,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":13240,"raft.queue_time_us":158233,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":38369,"thread_start_us":537345,"threads_started":3,"wal-append.queue_time_us":528253}
I20260709 13:04:54.821707 20509 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:54.821264 20520 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.869365 15162 rpcz_store.cc:276] 0709 13:04:53.743860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.744403 (+   543us) service_pool.cc:224] Handling call
0709 13:04:53.764137 (+ 19734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.764446 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.764496 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.783190 (+ 18694us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.043795 (+260605us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:54.043836 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.043909 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.868691 (+824782us) raft_consensus.cc:1685] finished
0709 13:04:54.868710 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.868884 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":205845,"threads_started":1,"wal-append.queue_time_us":411937}
I20260709 13:04:54.914319 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:54.611464 20821 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.776400 20756 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:54.828585 20364 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:54.881947 20779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.017s	sys 0.044s
I20260709 13:04:54.919588 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:54.922152 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:54.929373 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.005551 15232 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:04:54.911911 20820 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:04:54.937898 20844 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:54.938450 20682 consensus_queue.cc:1243] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:54.922031 20509 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:54.944094 20689 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.950985 20522 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.953392 20817 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:54.938566 20775 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:54.909698 20736 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:54.957549 20744 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.963745 20821 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:54.916515 20826 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:54.966014 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 236582us
I20260709 13:04:55.017936 20780 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:54.966634 20786 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:54.974048 20789 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.927s	user 0.013s	sys 0.000s
I20260709 13:04:54.960947 20761 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.056521 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.034534 20775 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:54.982942 20763 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.137s	user 0.010s	sys 0.002s
W20260709 13:04:55.000916 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.005s	sys 0.038s
I20260709 13:04:54.992740 20382 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:55.024700 15847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.004s	sys 0.033s
W20260709 13:04:55.004966 20779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.017s	sys 0.045s
I20260709 13:04:55.029922 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.033695 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:54.976341 20687 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.042388 15228 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:54.970283 15156 raft_consensus.cc:2368] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:55.047017 20837 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.043324 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 265392us
I20260709 13:04:55.050877 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.028731 20844 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.052457 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.023s	sys 0.046s
I20260709 13:04:55.054711 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2469) took 1221 ms. Trace:
W20260709 13:04:55.081835 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.023s	sys 0.047s
I20260709 13:04:55.063383 20775 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:54.994117 20746 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.008s	sys 0.025s
I20260709 13:04:55.040810 20736 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.062155 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.006s	sys 0.038s
W20260709 13:04:55.018836 15234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.028s	sys 0.046s
W20260709 13:04:55.064395 15847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.005s	sys 0.033s
W20260709 13:04:55.053189 20829 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:04:55.058888 20466 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.917s	user 0.008s	sys 0.004s
I20260709 13:04:55.047870 20522 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:55.069299 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 314127us
I20260709 13:04:55.105348 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.106076 15160 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:04:55.078361 15232 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:55.081874 15157 rpcz_store.cc:276] 0709 13:04:53.833227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.833402 (+   175us) service_pool.cc:224] Handling call
0709 13:04:53.865218 (+ 31816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.865590 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.865621 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.888369 (+ 22748us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.138331 (+249962us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:54.138391 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.138486 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.054396 (+915910us) raft_consensus.cc:1685] finished
0709 13:04:55.054426 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.054630 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":470,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":18539,"thread_start_us":137105,"threads_started":1,"wal-append.queue_time_us":832676}
I20260709 13:04:55.057463 15155 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:04:55.052731 20532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.005s	sys 0.009s
I20260709 13:04:55.086527 15154 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
W20260709 13:04:55.096830 15234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.029s	sys 0.046s
I20260709 13:04:55.097697 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1407) took 1421 ms. Trace:
W20260709 13:04:55.098093 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.100457 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.059460 20789 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.073359 20779 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.059368 20780 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:55.087110 20746 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.691s	user 0.008s	sys 0.025s
I20260709 13:04:55.111649 15154 pending_rounds.cc:85] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:04:55.111866 15847 rpcz_store.cc:276] 0709 13:04:53.676085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.676347 (+   262us) service_pool.cc:224] Handling call
0709 13:04:53.676586 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.676763 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:54.227137 (+550374us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.097316 (+870179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.097383 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:55.097432 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.097503 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.097638 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112950,"prepare.queue_time_us":260218,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":27,"raft.queue_time_us":231573,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":1286724,"threads_started":3,"wal-append.queue_time_us":800096}
I20260709 13:04:55.084458 20844 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.117861 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:55.119747 20839 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:55.120631 20806 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:55.122726 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:55.124071 20812 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.592s	user 0.005s	sys 0.002s
I20260709 13:04:55.124750 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.083000 20786 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.087317 20850 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.099428 20736 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:55.064421 20763 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.133227 15081 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:55.129500 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.127305 15154 pending_rounds.cc:107] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:55.190580 15154 raft_consensus.cc:2889] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:55.131862 20762 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.920s	user 0.004s	sys 0.008s
I20260709 13:04:55.138485 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.195182 15157 raft_consensus.cc:2410] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:55.141026 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.141592 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:55.137773 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.195928 15230 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:55.139681 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.143020 15840 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:04:55.192066 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.110836 20817 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:55.050127 20826 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.110944 20761 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:04:55.185868 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68077.7us
I20260709 13:04:55.136147 20745 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.118741 20522 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.127820 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:55.112717 20532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.006s	sys 0.009s
I20260709 13:04:55.185153 20752 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.133420 20746 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:55.109483 20846 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:55.128695 20756 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.144476 20839 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.242102 20840 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.198781 20856 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:55.213481 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 86203.3us
I20260709 13:04:55.205235 20762 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:55.153103 20812 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:55.231918 20848 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:04:55.224058 20846 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.234426 20834 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.239826 20825 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.000s
W20260709 13:04:55.240326 20838 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 13:04:55.240758 20746 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:55.241571 20788 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.042s	user 0.007s	sys 0.005s
I20260709 13:04:55.148078 20806 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.251121 20853 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:55.130987 20789 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.218773 20754 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:55.256443 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:55.317304 20754 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.317724 20754 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:55.317775 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61457us
I20260709 13:04:55.191103 20850 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.318645 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:55.319443 20364 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:55.320210 20364 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:55.320595 20364 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:55.210891 20817 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.218601 20532 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:55.220466 20681 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.129451 20779 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.278476 20827 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:55.320808 20850 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.278908 15080 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:55.323047 20827 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.279735 20839 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.282636 20844 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:55.283351 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1342) took 2259 ms (client timeout 1564 ms). Trace:
I20260709 13:04:55.286690 15230 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:55.216744 20761 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.297547 20763 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.267352 20786 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.299423 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2464) took 1084 ms. Trace:
I20260709 13:04:55.302913 15229 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:04:55.242668 20840 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.252764 20822 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.322024 15839 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:04:55.325577 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:55.321264 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:55.327456 15848 rpcz_store.cc:269] 0709 13:04:53.024012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.024936 (+   924us) service_pool.cc:224] Handling call
0709 13:04:55.283116 (+2258180us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54006ccfd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:55.283229 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576465664}
I20260709 13:04:55.274549 20509 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.331073 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.335223 15234 rpcz_store.cc:276] 0709 13:04:54.215059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.215362 (+   303us) service_pool.cc:224] Handling call
0709 13:04:54.229105 (+ 13743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.253612 (+ 24507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.253647 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.270365 (+ 16718us) log.cc:844] Serialized 26 byte log entry
0709 13:04:55.111935 (+841570us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:55.111978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.112046 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.299156 (+187110us) raft_consensus.cc:1685] finished
0709 13:04:55.299187 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.299353 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24114,"raft.queue_time_us":45879,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8836,"thread_start_us":730774,"threads_started":1,"wal-append.queue_time_us":747135}
I20260709 13:04:55.345523 15230 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:55.332157 20789 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.332119 20821 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:55.311946 20752 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.287328 20834 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.353629 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.338805 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:55.308993 20788 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.332885 20682 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:55.330121 20774 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.492s	user 0.006s	sys 0.009s
I20260709 13:04:55.321069 20364 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:55.330910 20466 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.305241 20825 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.322542 20681 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.347941 20456 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.402714 15841 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:04:55.391180 20859 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:04:55.411912 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:55.408166 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.414628 15230 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:55.416808 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.419757 15848 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:55.397262 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.355330 20821 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.424494 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.322278 20532 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.394469 20847 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:55.426990 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59417.3us
I20260709 13:04:55.334930 20853 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.354693 20789 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.447247 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.359539 20752 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.394127 20774 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.449983 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:55.398069 20682 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.396582 20466 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.427773 20780 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.459997 20842 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.004s
I20260709 13:04:55.464226 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.463480 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:55.471591 15157 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.454994 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.413579 20456 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.467479 15234 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:55.461057 20827 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.348106 20822 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.477240 20846 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.489477 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.483335 20834 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.486228 20853 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.494658 20461 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:55.494971 20761 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.495326 20461 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.452237 20522 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.497797 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:55.498512 15230 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:55.462102 20814 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:55.511274 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:55.512887 20814 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.513451 15157 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:04:55.509203 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79779.9us
I20260709 13:04:55.519743 15230 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:55.520853 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.473279 20847 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:04:55.480572 20826 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.508461 20844 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.531749 15848 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 10: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:55.534723 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.537297 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.538161 15157 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:55.477993 20842 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.530889 20812 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.546849 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:55.550828 20840 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.551872 20825 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.553336 20456 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.554499 20789 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.555122 20871 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:55.555572 15228 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:55.561301 15230 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:55.562685 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1338) took 2574 ms. Trace:
I20260709 13:04:55.562932 20846 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.565418 20466 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.559303 20461 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.566277 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.569648 15846 rpcz_store.cc:276] 0709 13:04:52.988123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.988278 (+   155us) service_pool.cc:224] Handling call
0709 13:04:53.004748 (+ 16470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.222707 (+217959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.562364 (+2339657us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54002a12d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:55.562569 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499459328}
W20260709 13:04:55.578055 15843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1349) took 1710 ms (client timeout 1756 ms). Trace:
I20260709 13:04:55.578159 20853 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:55.578660 15843 rpcz_store.cc:269] 0709 13:04:53.866566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.866759 (+   193us) service_pool.cc:224] Handling call
0709 13:04:55.566120 (+1699361us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:04:55.577076 (+ 10956us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9827584}
I20260709 13:04:55.586437 15230 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:55.566847 20871 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.576390 20846 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.588915 20681 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.614744 20690 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:55.646226 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.647087 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.654443 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.639211 20846 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.639966 20842 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.662078 20809 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.737s	user 0.000s	sys 0.012s
W20260709 13:04:55.662977 20775 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.005s
I20260709 13:04:55.639770 20682 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.673537 20871 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.640214 20754 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.675649 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.689213 15846 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:55.678943 20821 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.688664 20775 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.643589 20774 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.638312 20461 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:55.655655 20834 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:55.683307 20805 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.526s	user 0.024s	sys 0.015s
W20260709 13:04:55.684271 20766 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 2.294s	user 0.000s	sys 0.016s
I20260709 13:04:55.684183 20736 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.686802 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1341) took 1949 ms. Trace:
I20260709 13:04:55.701608 15086 rpcz_store.cc:276] 0709 13:04:53.737731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.751446 (+ 13715us) service_pool.cc:224] Handling call
0709 13:04:53.774817 (+ 23371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.002774 (+227957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.686597 (+1683823us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003250d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:55.686753 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348667136}
I20260709 13:04:55.702935 15160 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:55.703238 20814 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.687646 15234 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:55.685338 20809 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.704814 20738 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.652180 20752 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.640929 20364 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.706061 15228 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:55.653218 20747 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:04:55.710315 20747 consensus_queue.cc:1243] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:04:55.710675 20800 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.639s	user 0.004s	sys 0.001s
I20260709 13:04:55.710767 20809 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.694898 20775 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.698671 20834 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.712682 20364 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.696310 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44839.6us
I20260709 13:04:55.715005 20747 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.715175 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:55.715579 20747 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.716442 20834 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.721493 20754 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.723430 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:55.723467 20817 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:55.718206 20769 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 2.341s	user 0.008s	sys 0.012s
I20260709 13:04:55.718327 20747 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.734153 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2883) took 2801 ms. Trace:
W20260709 13:04:55.736946 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44743.1us
I20260709 13:04:55.734840 15087 rpcz_store.cc:276] 0709 13:04:52.929741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.942580 (+ 12839us) service_pool.cc:224] Handling call
0709 13:04:52.942922 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.981414 (+ 38492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.981462 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.001383 (+ 19921us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.196655 (+195272us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:53.196714 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.196809 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.717493 (+2520684us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:55.729949 (+ 12456us) raft_consensus.cc:1685] finished
0709 13:04:55.729983 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.730874 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54961,"raft.queue_time_us":58173,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":61914880,"thread_start_us":64393,"threads_started":1,"wal-append.queue_time_us":320463}
I20260709 13:04:55.744071 20776 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:55.747555 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:55.748304 20821 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:55.749553 20821 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.749965 20821 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:55.752501 20842 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:55.756083 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:55.756755 20747 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:55.759855 20747 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:55.761893 20747 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:55.756961 20776 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.763047 20776 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.733816 20736 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:55.751015 20251 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:55.767000 20251 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.770781 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:55.772248 15129 connection.cc:256] RPC call timeout handler was delayed by -1.2072252930000786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:55.774546 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.774617 15163 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:55.775220 15155 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:55.776392 20781 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:55.776929 20781 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.776994 20736 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:55.777835 20532 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:55.777684 20736 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.774721 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:55.776592 15160 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:55.778378 20532 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.779907 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.780511 20736 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:55.780066 20757 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 2.616s	user 0.014s	sys 0.000s
I20260709 13:04:55.781243 20532 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.784986 20799 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.713s	user 0.009s	sys 0.006s
I20260709 13:04:55.790700 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.791419 15160 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:04:55.794255 20609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.852s	user 0.007s	sys 0.008s
W20260709 13:04:55.795670 20609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.853s	user 0.007s	sys 0.008s
I20260709 13:04:55.797652 20251 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:55.798148 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.801355 20680 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.804387 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.810122 15163 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:55.810602 15163 pending_rounds.cc:85] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 10
I20260709 13:04:55.810788 15163 pending_rounds.cc:107] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:55.811103 15163 raft_consensus.cc:2889] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:55.812626 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:55.817539 20791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.672s	user 0.016s	sys 0.006s
W20260709 13:04:55.818011 20791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.677s	user 0.016s	sys 0.007s
W20260709 13:04:55.835464 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50860.6us
I20260709 13:04:55.835981 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:55.836167 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:55.836753 15155 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:55.851528 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.851790 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.852421 15088 raft_consensus.cc:2368] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:55.852558 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.852912 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.853475 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.854008 15087 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:55.854149 15088 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:55.852437 15084 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:55.856961 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.858309 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.858500 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.859328 15079 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:04:55.869658 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.870093 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:55.871876 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 141038us
I20260709 13:04:55.872982 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.873739 15080 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:55.874109 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.850519 20856 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:55.874879 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2466) took 1654 ms. Trace:
I20260709 13:04:55.834605 20806 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:55.876642 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.880390 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.877964 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.878393 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:55.882087 15079 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:55.855245 20754 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:55.858016 20723 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:55.875862 15236 rpcz_store.cc:276] 0709 13:04:54.220871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.220984 (+   113us) service_pool.cc:224] Handling call
0709 13:04:54.221264 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.221460 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:54.235162 (+ 13702us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.744242 (+1509080us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b180044dbe0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:04:55.874545 (+130303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.874615 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:55.874660 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.874723 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.874830 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13507,"prepare.run_cpu_time_us":633,"prepare.run_wall_time_us":15237,"raft.queue_time_us":1069,"spinlock_wait_cycles":6907264,"thread_start_us":130017,"threads_started":1,"wal-append.queue_time_us":1642716}
I20260709 13:04:55.778003 20781 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.886435 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:55.867689 20725 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.871596 20379 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:55.887481 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.900266 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.901547 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.883365 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.889703 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.889765 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:55.902568 15079 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:55.881637 20378 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:04:55.896903 20725 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.896948 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:55.857530 20772 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:04:55.854916 20680 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:04:55.902596 15087 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:55.881557 20584 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:55.871176 20817 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:04:55.876006 20771 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:55.910064 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:55.879174 20856 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.894972 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.947978 15163 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:55.881899 20720 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:04:55.920149 20772 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:55.920467 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:55.921923 20725 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.923394 20379 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:55.923655 20821 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.923964 20747 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.892971 20754 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.924736 20680 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:55.926872 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.903155 15155 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:55.962983 15080 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
W20260709 13:04:55.938012 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.953298 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:55.965510 15087 raft_consensus.cc:2368] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:55.887777 20814 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 13:04:55.954530 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48492.4us
W20260709 13:04:55.958397 20879 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:04:55.950428 20856 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.961016 20754 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:55.970597 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.973949 15155 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:55.973831 20379 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:55.974754 20783 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 2.270s	user 0.009s	sys 0.004s
I20260709 13:04:55.949676 20772 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.976502 20772 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:55.972919 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:55.898808 20723 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.977840 20772 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:55.989266 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:55.949577 20584 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.961870 20680 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:55.990150 15080 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:55.953848 20771 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:55.987386 20777 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.000864 20806 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:56.033316 20822 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.035794 15160 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:56.045166 15159 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:56.007809 20777 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:56.050127 15158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2896) took 2537 ms (client timeout 1687 ms). Trace:
W20260709 13:04:56.050518 15158 rpcz_store.cc:269] 0709 13:04:53.507917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.509155 (+  1238us) service_pool.cc:224] Handling call
0709 13:04:56.044689 (+2535534us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b54002882d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.044869 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5120063872}
I20260709 13:04:56.029162 20806 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.029590 20680 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:56.060055 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.065135 20423 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.065165 20821 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.188566 15848 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:56.076546 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.200469 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.309290 15158 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.093374 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.303524 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.080865 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.101836 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:56.098134 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2902) took 2327 ms (client timeout 1643 ms). Trace:
W20260709 13:04:56.312083 15156 rpcz_store.cc:269] 0709 13:04:53.770821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.771848 (+  1027us) service_pool.cc:224] Handling call
0709 13:04:56.097896 (+2326048us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400085cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.098059 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2698283264}
I20260709 13:04:56.080396 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:56.081252 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.313663 15160 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:56.095906 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.306363 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.314733 15234 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:56.209713 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:56.313673 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 233369us
I20260709 13:04:56.099318 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.101507 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.207533 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2254) took 2006 ms. Trace:
I20260709 13:04:56.316972 15231 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:56.317133 15081 rpcz_store.cc:276] 0709 13:04:54.201115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.205433 (+  4318us) service_pool.cc:224] Handling call
0709 13:04:56.104616 (+1899183us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54000dbbd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.207449 (+102833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131246464}
I20260709 13:04:56.317549 15846 raft_consensus.cc:2410] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:56.316367 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:56.319114 15162 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:56.311568 15840 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:04:56.322674 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:56.323262 15840 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:56.323855 15154 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:56.102097 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.100668 15843 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:56.328959 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.329375 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.331295 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.332080 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.332777 15156 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:56.333287 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:56.334457 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.334599 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.275738 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.338152 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:56.338740 15228 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:56.338907 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:56.226752 20898 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:04:56.226008 20897 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:04:56.340889 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.097537 20747 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:56.342677 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 67061us
I20260709 13:04:56.340862 15237 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:56.086725 20680 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:56.068724 20717 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.345748 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.100435 20584 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.091269 20777 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.348026 15236 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:56.328559 20289 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:56.349849 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:56.361732 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:56.341948 15156 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:56.292968 20609 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:56.083909 20883 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.099915 20806 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.351425 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.343394 20796 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:56.354416 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.362025 15160 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:04:56.345151 20784 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:56.363327 20609 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:56.347506 20773 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:56.366906 20871 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:56.368467 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.081801 20615 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.269s
I20260709 13:04:56.370388 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.088147 20723 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.370755 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.394502 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.356051 20797 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.369197 20289 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.419293 15843 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:04:56.418303 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.370232 15846 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:56.420289 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.430872 15228 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:56.419888 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:56.435115 20771 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.436527 20788 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.432623 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:56.394917 20045 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:56.418882 20722 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:56.420673 20797 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.420197 20894 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.415589 20871 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:56.444991 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.417189 20602 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:56.446910 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.421638 20289 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.432689 20796 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:56.450875 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.392920 20883 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.446678 20722 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:04:56.458189 20609 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.440580 20723 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:56.464511 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.447671 20784 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:56.455477 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.451948 20797 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:56.483281 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.483695 15839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1406) took 1360 ms (client timeout 1742 ms). Trace:
I20260709 13:04:56.442945 20045 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:56.461773 20602 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:56.484082 15839 rpcz_store.cc:269] 0709 13:04:55.122958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.129245 (+  6287us) service_pool.cc:224] Handling call
0709 13:04:56.483490 (+1354245us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540036db54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.483620 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302668672}
I20260709 13:04:56.473047 20722 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.492659 15082 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:04:56.494406 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.491521 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.496522 15156 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:56.497689 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.471148 20796 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:56.475996 20723 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.570434 15158 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:56.503634 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.505419 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.505959 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.461402 20894 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.508942 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:56.487432 20423 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.489451 20602 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:56.519707 20901 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:04:56.519587 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.474534 20856 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.484344 20784 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:04:56.531252 15229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2192) took 2321 ms (client timeout 2059 ms). Trace:
I20260709 13:04:56.535806 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1409) took 1412 ms. Trace:
I20260709 13:04:56.682727 20784 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:56.682837 15841 rpcz_store.cc:276] 0709 13:04:55.123845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.130611 (+  6766us) service_pool.cc:224] Handling call
0709 13:04:55.130918 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.404514 (+273596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.535520 (+1131006us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540049c0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:56.535727 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624226560}
W20260709 13:04:56.682765 15229 rpcz_store.cc:269] 0709 13:04:54.209444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.210477 (+  1033us) service_pool.cc:224] Handling call
0709 13:04:56.530980 (+2320503us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540043e754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.531119 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454651776}
I20260709 13:04:56.684332 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.537699 20612 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.501340 20871 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:56.549914 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.548600 20289 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.577647 15839 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:56.540717 20532 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.695766 20532 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:56.696305 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 176842us
I20260709 13:04:56.697089 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.697966 20532 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.489115 20045 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.698589 15052 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:56.699786 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:56.499668 20609 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.700919 20609 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.701397 20856 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.702025 20856 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:56.702116 20680 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.702539 20680 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.702696 20856 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:56.702941 20680 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.702999 20646 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:56.703341 20289 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.665309 15083 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:04:56.670924 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:56.704212 15083 pending_rounds.cc:85] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 11
I20260709 13:04:56.704504 15083 pending_rounds.cc:107] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:04:56.601008 20723 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:56.724656 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:04:56.676409 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 112207us
I20260709 13:04:56.694646 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.704784 15083 raft_consensus.cc:2889] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:56.709390 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.729099 15082 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:56.714941 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.718894 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.731166 15230 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:56.722808 15233 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:56.734841 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.735157 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.735618 15086 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:04:56.736370 15087 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:04:56.737675 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:56.738444 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67650us
I20260709 13:04:56.740557 15846 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:56.743340 15229 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:56.738695 15845 raft_consensus.cc:2368] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
I20260709 13:04:56.747761 15840 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:04:56.726275 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.752902 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:56.754428 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2894) took 3247 ms (client timeout 1546 ms). Trace:
I20260709 13:04:56.757018 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.734581 20609 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.647145 20790 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:04:56.669857 20423 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:56.763336 20721 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.763985 20721 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.764920 20423 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.765825 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.766595 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.766974 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:56.767783 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.771024 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.772078 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.671090 20883 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.773236 15081 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:56.774631 20894 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:56.775372 20787 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:56.776162 20787 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:56.776495 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.752251 20580 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:56.777120 15088 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.777925 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.778657 15157 raft_consensus.cc:2368] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:56.780036 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.780730 15079 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:56.780809 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:56.752010 20722 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.781211 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.781473 15840 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.781155 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.782398 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.779623 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.775313 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.778576 20787 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:56.783918 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:56.786458 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:56.781571 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:56.754997 15161 rpcz_store.cc:269] 0709 13:04:53.507258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.507431 (+   173us) service_pool.cc:224] Handling call
0709 13:04:56.754201 (+3246770us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54002baf54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:56.754351 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4549099392}
I20260709 13:04:56.782305 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.787608 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.788275 15235 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:56.780133 20289 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:56.780045 20795 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:56.713900 20796 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.793622 15848 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:56.793623 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.794281 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.795389 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.797533 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.798676 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:56.790659 20609 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.805387 15079 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:56.803992 15088 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:56.807981 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.805902 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.809587 15080 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.810258 15082 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:56.821631 15236 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:56.822993 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.823344 15840 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:56.835959 15234 pending_rounds.cc:85] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 7
I20260709 13:04:56.836524 15234 pending_rounds.cc:107] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:04:56.836876 15234 raft_consensus.cc:2889] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:56.792116 20795 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:56.741829 20791 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.791826 20790 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:56.841329 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.855283 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.856484 20784 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.857236 20646 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:56.857739 20646 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.857525 20795 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.807403 20289 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:56.859704 20289 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:56.850402 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55219.8us
I20260709 13:04:56.841883 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.824740 15236 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:56.858687 20784 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:56.841712 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:56.867082 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.863526 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.867743 15236 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:56.865270 15086 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:04:56.850392 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49209.4us
I20260709 13:04:56.855283 15237 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:56.846146 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:56.873306 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.874130 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:56.874372 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:56.874624 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.875820 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:56.876083 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:56.876523 15231 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.874430 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.876053 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.828264 20609 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:56.888571 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.881335 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:56.881498 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.033s	sys 0.039s
I20260709 13:04:56.882922 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:56.892442 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.033s	sys 0.039s
I20260709 13:04:56.786352 20841 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:04:56.895709 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.888468 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:56.892803 15847 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:56.898099 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.828550 20532 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:56.899995 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.900589 15846 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:56.871400 20646 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:56.895180 15236 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:56.863265 20790 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:56.879647 20781 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:56.892971 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 539881us
I20260709 13:04:56.907369 15157 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:56.908313 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.893695 20609 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.922097 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:56.921248 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.925802 15203 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:56.922705 20717 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:56.948033 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:56.978237 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 52567.3us
I20260709 13:04:56.882999 20804 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:56.871918 20791 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:56.995375 20791 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:04:56.954658 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 78648.7us
I20260709 13:04:56.889623 20835 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:04:56.996475 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:56.996615 20835 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:56.997189 20835 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:56.998145 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.999259 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.999326 15233 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:04:56.999830 15228 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:57.000093 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.001453 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.002000 15155 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.959930 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:56.960340 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.004446 15847 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:57.005016 20814 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:56.964277 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.008522 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:57.009183 15844 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.009248 15158 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:57.010035 20870 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.010557 20870 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.964685 20796 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:56.950758 20784 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.013199 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2470) took 3179 ms. Trace:
I20260709 13:04:57.013671 15159 rpcz_store.cc:276] 0709 13:04:53.833843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.833984 (+   141us) service_pool.cc:224] Handling call
0709 13:04:56.044951 (+2210967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.049248 (+  4297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.012960 (+963712us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54002882d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:57.013117 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5026125184}
I20260709 13:04:57.015591 15159 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:56.965394 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 2892) took 3211 ms. Trace:
I20260709 13:04:56.999336 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:57.017231 15083 rpcz_store.cc:276] 0709 13:04:53.754071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.754236 (+   165us) service_pool.cc:224] Handling call
0709 13:04:53.770288 (+ 16052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.753794 (+2983506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.965166 (+211372us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b54001b0dd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:56.965338 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6656709888}
W20260709 13:04:56.967931 15162 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.038s	sys 0.065s
I20260709 13:04:56.947064 20894 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:57.030864 15159 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:56.973529 15202 leader_election.cc:400] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:56.841039 20680 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.975504 15086 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:04:56.979974 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:57.033620 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 53744us
I20260709 13:04:56.980937 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:56.981236 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:56.983091 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:56.984925 20604 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:56.985828 20903 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.157s	user 0.003s	sys 0.000s
I20260709 13:04:56.986285 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:56.988384 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.002146 15233 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:57.057670 15161 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.037995 15082 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:57.059196 15081 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:56.991859 20602 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.013509 20784 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.957144 20785 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.019078 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:57.019829 15162 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.038s	sys 0.065s
I20260709 13:04:56.972607 20781 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.086846 20781 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.088598 15236 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:57.025240 20379 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.009285 20790 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.031071 15228 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:57.095824 15085 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:57.031482 15202 leader_election.cc:403] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:57.096895 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 123555us
I20260709 13:04:57.097790 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:57.099414 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:57.100646 15080 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:56.935572 15231 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:04:57.068140 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49168us
I20260709 13:04:57.070730 20784 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.071667 15842 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:57.067377 20841 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.110948 15202 leader_election.cc:400] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:57.073267 20602 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.075814 20723 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:56.998193 20835 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.117103 20835 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:57.118261 20680 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:57.031749 20894 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.121562 20721 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:57.084859 20847 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:57.123001 20847 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:57.123440 20847 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:57.085232 20646 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:57.083498 20609 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.086046 20787 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:57.123607 20894 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.123543 20580 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:57.106554 15847 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:04:57.110087 15844 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:04:57.121707 20797 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.000s
I20260709 13:04:57.028882 20796 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.121479 15841 raft_consensus.cc:2368] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:04:57.121366 15161 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:04:57.135460 20790 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:57.105347 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:57.126551 20604 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:57.179548 15159 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:57.186902 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81621.4us
I20260709 13:04:57.188382 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:57.189631 20602 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:57.191993 20602 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:57.193699 20784 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:57.194239 20784 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:57.195616 20784 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.199055 20784 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:57.181895 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.182754 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.205358 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.205780 15847 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:57.206223 15839 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:57.207077 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.208057 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.208683 15839 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:57.116313 20045 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.110913 20791 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
I20260709 13:04:57.124761 15155 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:57.208784 20721 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:04:57.179879 15163 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.021s	sys 0.027s
I20260709 13:04:57.095307 20379 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:57.219230 15163 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.021s	sys 0.027s
I20260709 13:04:57.214969 20894 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.185387 20797 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.252871 15843 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:57.255354 20797 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:57.216005 20907 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:57.257193 20907 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.257364 15847 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:57.141552 20787 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.257642 20907 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:57.269757 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.270270 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.270912 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.275125 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:57.214594 20604 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:57.260587 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.263357 20609 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:57.268304 15154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.013s	sys 0.033s
I20260709 13:04:57.275852 15839 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:57.283356 15160 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.010s	sys 0.021s
W20260709 13:04:57.289228 15160 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.010s	sys 0.021s
I20260709 13:04:57.289999 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:57.273386 20615 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.224731 20721 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:57.291227 15154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.014s	sys 0.033s
I20260709 13:04:57.287817 20604 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.258530 15847 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:57.287423 20894 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:57.317442 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.320570 15845 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 10.
I20260709 13:04:57.300714 20609 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:57.302945 20612 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:57.340004 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.293324 20785 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:57.293066 20875 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:04:57.344961 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.300089 20689 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:04:57.326757 20894 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:57.301146 20615 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.335834 20609 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:57.337318 20612 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:57.345870 20723 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:57.350371 20785 consensus_queue.cc:1243] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:57.402132 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.359421 20894 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.366639 20612 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:57.404987 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:57.405597 15231 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:57.362605 20609 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:57.417621 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:57.419476 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.036s
W20260709 13:04:57.419821 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.000s	sys 0.036s
I20260709 13:04:57.418452 15845 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:04:57.421373 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.017s	sys 0.016s
W20260709 13:04:57.421936 15052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.024s
I20260709 13:04:57.415793 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:57.422266 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.003s	sys 0.024s
W20260709 13:04:57.423013 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 527354us
I20260709 13:04:57.423765 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:57.416646 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.009s	sys 0.021s
W20260709 13:04:57.425264 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.009s	sys 0.021s
W20260709 13:04:57.425637 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 626986us
I20260709 13:04:57.426524 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:57.422199 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.017s	sys 0.016s
I20260709 13:04:57.429814 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:57.433236 15231 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:57.432693 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.441352 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:57.449050 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:57.462566 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:57.430683 20835 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:57.429301 20906 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.481431 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:57.487620 15231 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.482613 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:57.436685 20777 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:57.443555 20045 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.489727 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:57.477427 20772 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.435212 20834 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.009s	sys 0.019s
W20260709 13:04:57.533372 20834 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.009s	sys 0.019s
I20260709 13:04:57.534937 20721 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:57.535771 20721 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:57.537365 20747 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:57.538060 20747 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:57.539139 20747 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.548115 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1369) took 1036 ms. Trace:
I20260709 13:04:57.550935 20870 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.554306 20722 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:57.546229 20615 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.520669 20787 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:57.536563 20721 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:57.556735 15847 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:57.558768 15847 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:57.561923 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1360) took 1072 ms. Trace:
I20260709 13:04:57.562381 15841 rpcz_store.cc:276] 0709 13:04:56.489689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.766756 (+277067us) service_pool.cc:224] Handling call
0709 13:04:57.561722 (+794966us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54004a2754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:57.561860 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":813543296}
I20260709 13:04:57.556452 20721 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:57.565959 20721 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:57.566711 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 532432us
I20260709 13:04:57.580946 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:57.574316 20772 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.587338 20772 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.587883 20847 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.492424 20906 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.482333 20379 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:57.521238 20777 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.548688 15845 rpcz_store.cc:276] 0709 13:04:56.511194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.351641 (+840447us) service_pool.cc:224] Handling call
0709 13:04:57.363390 (+ 11749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.420646 (+ 57256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.547779 (+127133us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54006d6fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:57.548030 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91405568}
I20260709 13:04:57.597893 15231 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:57.608865 20777 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:57.610107 20847 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:57.601809 20781 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:04:57.617692 15228 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:04:57.613425 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.024s	sys 0.034s
W20260709 13:04:57.628508 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.024s	sys 0.034s
W20260709 13:04:57.629055 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 745052us
W20260709 13:04:57.613708 20846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.015s	sys 0.029s
W20260709 13:04:57.614566 20532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.011s	sys 0.041s
W20260709 13:04:57.635027 15234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.022s	sys 0.025s
W20260709 13:04:57.635587 15234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.022s	sys 0.025s
I20260709 13:04:57.611642 20612 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:57.634500 20532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.012s	sys 0.041s
W20260709 13:04:57.634645 20846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.015s	sys 0.029s
W20260709 13:04:57.652256 15158 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.028s	sys 0.020s
W20260709 13:04:57.651237 15162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.025s	sys 0.051s
W20260709 13:04:57.654429 15162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.025s	sys 0.051s
I20260709 13:04:57.654471 15839 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:57.653270 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:57.659946 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 170283us
I20260709 13:04:57.660732 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:57.661906 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:57.654038 15158 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.028s	sys 0.020s
I20260709 13:04:57.663785 20386 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:57.664327 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.666266 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.669126 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.669528 15845 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:57.671212 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.671267 20721 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.674266 15085 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:04:57.676725 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.677522 15085 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:57.677860 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.680119 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.681320 15085 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.681568 20787 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:57.691325 20787 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.694130 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.694972 15084 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
I20260709 13:04:57.695873 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.696636 15080 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:57.698510 15085 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:57.703517 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.704201 15080 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:57.691912 20787 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.705929 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:57.706095 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:57.708753 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.708726 20386 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:57.717736 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.718160 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 291683us
I20260709 13:04:57.719628 20835 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:57.720676 20835 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.721690 15848 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:57.724128 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:57.616767 20781 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.659507 20856 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:57.729283 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 291536us
I20260709 13:04:57.730360 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1351) took 1314 ms. Trace:
I20260709 13:04:57.734066 15843 rpcz_store.cc:276] 0709 13:04:56.416318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.439994 (+ 23676us) service_pool.cc:224] Handling call
0709 13:04:56.440325 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.253376 (+813051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.730104 (+476728us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:57.730285 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868091136}
I20260709 13:04:57.734841 15127 leader_election.cc:400] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
W20260709 13:04:57.732122 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 304427us
I20260709 13:04:57.735170 15127 leader_election.cc:403] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:57.717640 15080 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:57.649438 20532 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:57.735610 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:57.737247 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.746889 15088 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:57.736507 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.748492 15085 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.747373 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.747458 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.754068 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.754963 15083 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:57.651849 20772 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:04:57.759812 15082 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:57.736764 20386 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.763938 20918 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:04:57.765786 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.737430 20920 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.769225 20920 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.753039 20915 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.756961 20856 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:57.779834 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.780619 20906 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:57.768347 20790 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.005s
I20260709 13:04:57.785521 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:57.787292 20612 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.789716 15844 raft_consensus.cc:2410] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:04:57.792024 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:57.792025 15085 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:04:57.792968 15087 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:04:57.782887 20721 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:57.785941 20915 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.801117 20790 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:57.802865 20790 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:57.803315 20790 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:57.806018 20795 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:57.800575 20915 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:57.782122 20920 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.789649 20580 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:04:57.812515 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.813153 15161 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.814881 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.804173 20790 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.819345 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.770546 20386 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.822113 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:57.796134 20916 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:57.838224 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1425) took 1411 ms. Trace:
I20260709 13:04:57.838622 15842 rpcz_store.cc:276] 0709 13:04:56.427419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.427578 (+   159us) service_pool.cc:224] Handling call
0709 13:04:57.837990 (+1410412us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:57.838155 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420251008}
I20260709 13:04:57.840037 15079 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:57.836640 15156 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.853796 15161 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:57.839066 20609 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:57.873585 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.864439 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:04:57.881958 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.809995 20913 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.883899 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2484) took 1163 ms. Trace:
I20260709 13:04:57.811185 20915 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
I20260709 13:04:57.897003 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:57.900440 15157 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 9.
I20260709 13:04:57.903831 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2501) took 1017 ms. Trace:
I20260709 13:04:57.904472 15158 rpcz_store.cc:276] 0709 13:04:56.886127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.010257 (+124130us) service_pool.cc:224] Handling call
0709 13:04:57.010598 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.664182 (+653584us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:57.664532 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.903220 (+238688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.903442 (+   222us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:57.903503 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.903602 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.903758 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6296,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":761132,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8028,"thread_start_us":891237,"threads_started":2}
I20260709 13:04:57.796456 20935 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 13:04:57.846004 20916 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:57.857849 20920 raft_consensus.cc:2764] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:57.911089 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:57.913300 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:57.897341 20646 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.939877 15161 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:57.809722 20936 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:04:57.902979 20846 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:57.821936 20532 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.904798 20790 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.951682 15156 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
W20260709 13:04:57.952006 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 233702us
I20260709 13:04:57.955981 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2911) took 1465 ms. Trace:
I20260709 13:04:57.963020 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:57.947904 20847 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:57.971603 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:57.960736 15160 rpcz_store.cc:276] 0709 13:04:56.490365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.490524 (+   159us) service_pool.cc:224] Handling call
0709 13:04:56.491731 (+  1207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.491920 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:57.289674 (+797754us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.955484 (+665810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.955557 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:57.955651 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.955811 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.955924 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16075,"prepare.queue_time_us":206181,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":110278,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":12266,"thread_start_us":1401268,"threads_started":3,"wal-append.queue_time_us":618758}
I20260709 13:04:57.907289 20856 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.941627 20646 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:57.976815 20847 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.016031 20847 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:57.966269 20777 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:57.998504 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:58.019879 20777 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.020519 20777 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:58.016712 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45230us
I20260709 13:04:57.995522 20781 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.026535 20646 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:57.995831 20916 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.026587 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2482) took 1307 ms. Trace:
I20260709 13:04:58.043489 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:57.995159 20937 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:58.047079 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.047703 15233 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.030478 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1365) took 1159 ms. Trace:
I20260709 13:04:58.031284 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:57.884699 15232 rpcz_store.cc:276] 0709 13:04:56.720163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.795074 (+ 74911us) service_pool.cc:224] Handling call
0709 13:04:56.856006 (+ 60932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.856257 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:57.662444 (+806187us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.883452 (+221008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.883528 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:57.883591 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.883669 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.883823 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366271,"prepare.queue_time_us":806198,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":30,"raft.queue_time_us":13883,"thread_start_us":659611,"threads_started":2}
I20260709 13:04:58.048441 15081 rpcz_store.cc:276] 0709 13:04:56.871098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.874502 (+  3404us) service_pool.cc:224] Handling call
0709 13:04:57.058969 (+184467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.084435 (+ 25466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.030277 (+945842us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400305f50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.030416 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372870528}
I20260709 13:04:58.044351 15235 rpcz_store.cc:276] 0709 13:04:56.719608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.789113 (+ 69505us) service_pool.cc:224] Handling call
0709 13:04:56.842086 (+ 52973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:56.842568 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:57.203601 (+361033us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18000d54c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:04:57.662867 (+459266us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.865099 (+202232us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.903513 (+ 38414us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.903669 (+   156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:57.903715 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.903811 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.026186 (+122375us) raft_consensus.cc:1685] finished
0709 13:04:58.026315 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.026509 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36952,"prepare.queue_time_us":819693,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":10835,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":26196,"spinlock_wait_cycles":5873920,"thread_start_us":659233,"threads_started":2,"wal-append.queue_time_us":361325}
I20260709 13:04:58.033378 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.051395 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:58.034837 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:58.036100 20922 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:04:57.956612 20846 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:58.009243 20604 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.000s
W20260709 13:04:57.989598 20927 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:58.072217 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:58.061322 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 325779us
W20260709 13:04:58.007345 20907 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.000s
I20260709 13:04:58.064816 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.064888 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.067902 15086 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:58.077416 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1293) took 1182 ms. Trace:
I20260709 13:04:58.078624 15231 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:58.078966 15159 rpcz_store.cc:276] 0709 13:04:56.894854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.077731 (+182877us) service_pool.cc:224] Handling call
0709 13:04:57.078097 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.214008 (+135911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.077046 (+863038us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002878d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:58.077327 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232838528}
W20260709 13:04:58.082466 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 347912us
I20260709 13:04:58.084326 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.105294 15160 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:04:58.062032 20906 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:58.087956 15128 leader_election.cc:400] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:04:58.109846 15128 leader_election.cc:403] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:58.063347 20870 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:58.063000 20423 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:58.086870 20723 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:58.087141 20907 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.086601 20722 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:58.093708 15848 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:58.099546 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.121794 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.099607 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.120671 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.125411 15159 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:58.108461 15233 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:04:58.110450 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:58.111420 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2495) took 2032 ms. Trace:
I20260709 13:04:58.105696 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.121838 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1434) took 1611 ms. Trace:
I20260709 13:04:58.128181 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.129238 15839 rpcz_store.cc:276] 0709 13:04:56.509930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.289684 (+779754us) service_pool.cc:224] Handling call
0709 13:04:57.303529 (+ 13845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.658871 (+355342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.121497 (+462626us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400498c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:58.121757 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806633856}
I20260709 13:04:58.132001 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.133191 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:58.133855 15158 raft_consensus.cc:2368] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:58.133759 20871 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.134558 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.134727 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:58.135558 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:58.135848 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:58.136173 15159 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:04:58.137470 20423 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:58.137953 20423 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.139566 20954 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:58.140071 20954 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:58.140476 20954 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:58.127669 15154 rpcz_store.cc:276] 0709 13:04:56.078827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.337135 (+258308us) service_pool.cc:224] Handling call
0709 13:04:56.357378 (+ 20243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:56.358393 (+  1015us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:56.530186 (+171793us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b18002c5720. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:04:57.301737 (+771551us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.053256 (+751519us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.053790 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.053970 (+   180us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:58.054019 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.054131 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.097630 (+ 43499us) raft_consensus.cc:1685] finished
0709 13:04:58.111155 (+ 13525us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.111341 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41488,"prepare.queue_time_us":338991,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":115307,"raft.run_cpu_time_us":7130,"raft.run_wall_time_us":50343,"spinlock_wait_cycles":54680832,"thread_start_us":1481339,"threads_started":3,"wal-append.queue_time_us":477831}
I20260709 13:04:58.141800 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:58.144117 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:58.144323 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2494) took 2065 ms. Trace:
I20260709 13:04:58.145756 20612 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:58.146242 20612 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.146631 20612 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:58.148140 20870 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:58.148496 20723 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:58.141017 20923 raft_consensus.cc:2764] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:58.133770 15848 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:58.144773 15162 rpcz_store.cc:276] 0709 13:04:56.078541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.334814 (+256273us) service_pool.cc:224] Handling call
0709 13:04:56.348884 (+ 14070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.438348 (+ 89464us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:57.067897 (+629549us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.908449 (+840552us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.908543 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:57.908626 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.144100 (+235474us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:58.144151 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.144270 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79228,"prepare.queue_time_us":259365,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":27,"raft.queue_time_us":150431,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":205192576,"thread_start_us":1381801,"threads_started":3,"wal-append.queue_time_us":849340}
I20260709 13:04:58.148613 20870 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:58.151737 20870 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:58.152243 20680 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:58.153381 15840 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:58.141852 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.155109 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1368) took 1644 ms. Trace:
I20260709 13:04:58.155453 15840 rpcz_store.cc:276] 0709 13:04:56.510895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.841398 (+330503us) service_pool.cc:224] Handling call
0709 13:04:58.154944 (+1313546us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54006f0254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.155058 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2943586560}
I20260709 13:04:58.156946 20846 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.157641 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.158706 20871 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.158789 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:58.159716 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:58.151837 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.078593 20604 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.113622 20906 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.160987 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:58.169287 20906 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:58.161888 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.169857 20957 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:58.170639 20957 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:58.171029 20957 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:58.171108 20797 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:58.171496 20957 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.171805 20797 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:58.172879 20787 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:58.172812 20797 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.173569 20580 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:58.175629 20604 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.163501 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.176299 15236 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 10.
W20260709 13:04:58.119344 20894 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.006s
I20260709 13:04:58.177317 15154 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:58.177139 20894 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.178352 20907 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.179334 20894 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.090346 20956 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:58.180163 20956 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.159765 20423 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.182477 20956 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.182844 20423 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.113895 20834 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:58.169900 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44877.4us
I20260709 13:04:58.183248 15846 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:58.169411 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.189467 15158 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:58.201594 15162 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:58.164974 20871 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:58.183308 20788 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:04:58.121382 20955 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:04:58.168941 20804 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:58.210264 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:58.211782 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2912) took 1721 ms. Trace:
W20260709 13:04:58.211014 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43891.6us
I20260709 13:04:58.216907 15128 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.216048 15163 rpcz_store.cc:276] 0709 13:04:56.490654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.490753 (+    99us) service_pool.cc:224] Handling call
0709 13:04:56.491162 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.491361 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:57.220915 (+729554us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.190572 (+969657us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.211293 (+ 20721us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.211460 (+   167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:58.211501 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.211554 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.211623 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.211728 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97722,"prepare.queue_time_us":213477,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":85,"raft.queue_time_us":171550,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"spinlock_wait_cycles":269056,"thread_start_us":1618002,"threads_started":3,"wal-append.queue_time_us":617602}
I20260709 13:04:58.222497 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:58.226585 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.160537 20920 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:04:58.229802 20955 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:58.236563 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:58.165179 20722 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.149674 20723 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.245558 20386 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.248879 20956 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:58.249023 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1433) took 1739 ms. Trace:
I20260709 13:04:58.258399 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:58.253464 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2485) took 1533 ms. Trace:
I20260709 13:04:58.273201 15839 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:58.254508 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.275337 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:58.275588 15233 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:58.276450 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.266685 15159 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:04:58.278059 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:58.272619 15844 rpcz_store.cc:276] 0709 13:04:56.509656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.181551 (+671895us) service_pool.cc:224] Handling call
0709 13:04:58.248762 (+1067211us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b54006cdc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.248957 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343222144}
I20260709 13:04:58.255357 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.280053 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.280732 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.282555 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.262130 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:58.286996 15228 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:58.289674 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 241544us
I20260709 13:04:58.273550 15234 rpcz_store.cc:276] 0709 13:04:56.720431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.802708 (+ 82277us) service_pool.cc:224] Handling call
0709 13:04:56.803070 (+   362us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:56.879829 (+ 76759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.879882 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:56.880981 (+  1099us) log.cc:844] Serialized 78 byte log entry
0709 13:04:57.636154 (+755173us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:58.252282 (+616128us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b1800378700. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf
0709 13:04:58.252439 (+   157us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.252875 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.253065 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.253136 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.253263 (+   127us) raft_consensus.cc:1685] finished
0709 13:04:58.253277 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.253409 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29401,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":14036,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":11467,"spinlock_wait_cycles":5759232,"thread_start_us":915752,"threads_started":2,"wal-append.queue_time_us":959081}
I20260709 13:04:58.287914 15158 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:04:58.293285 15157 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:58.289399 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.256174 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:58.294986 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.295325 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:58.295644 15234 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:04:58.296793 15203 leader_election.cc:400] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:58.299943 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.300983 15203 leader_election.cc:403] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:58.301630 15234 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:58.196947 20957 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.278998 20604 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.304656 20856 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.284623 20788 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:58.305788 20957 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.309305 20795 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:58.309851 20795 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:58.310887 20795 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.312129 20795 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.261102 20722 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.221794 15841 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:58.317574 15841 raft_consensus.cc:606] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:58.318045 15841 raft_consensus.cc:614] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:58.319012 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1431) took 1023 ms. Trace:
I20260709 13:04:58.319543 15846 rpcz_store.cc:276] 0709 13:04:57.295978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.344628 (+ 48650us) service_pool.cc:224] Handling call
0709 13:04:58.318710 (+974082us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b540041cdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.318938 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917792128}
I20260709 13:04:58.320801 15847 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:58.321496 20847 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.322705 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1435) took 1398 ms. Trace:
I20260709 13:04:58.323004 15848 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:58.323215 15847 rpcz_store.cc:276] 0709 13:04:56.923974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.559681 (+635707us) service_pool.cc:224] Handling call
0709 13:04:58.320561 (+760880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.321351 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.322442 (+  1091us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b540041cdd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.322630 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749004672}
I20260709 13:04:58.205757 20423 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
W20260709 13:04:58.285130 20927 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:04:58.331830 20906 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.340122 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.332962 20956 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:58.336212 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74128us
I20260709 13:04:58.261842 20955 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:58.337857 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81897.5us
I20260709 13:04:58.343940 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.259164 20920 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:58.338039 20957 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:58.338447 20923 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:58.340797 15234 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:58.341403 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46156.8us
W20260709 13:04:58.341370 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44760.9us
I20260709 13:04:58.367465 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.366271 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.372493 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.366336 15228 pending_rounds.cc:85] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:04:58.368986 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.373528 15228 pending_rounds.cc:107] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:58.374701 15228 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:58.292742 20781 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:58.387575 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.388408 15233 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:58.390156 15234 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:58.352330 20856 raft_consensus.cc:2764] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:58.342646 20791 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:58.386198 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.264256 20612 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.351104 20604 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:58.349576 20955 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.350304 20920 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:58.367373 20923 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.367246 20423 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:58.407395 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.392096 20957 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:58.384487 20788 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:58.402760 20936 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.400198 20680 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:58.400949 20950 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:58.404112 20847 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.367774 20959 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:04:58.401611 20791 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:58.406020 20955 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:58.408354 20923 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.426672 20906 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:58.427609 20906 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:04:58.410135 20917 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.406641 20870 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.395040 20781 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.403002 20717 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:04:58.403349 20646 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:04:58.418495 20907 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:58.434348 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2483) took 1710 ms. Trace:
W20260709 13:04:58.432363 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 267265us
I20260709 13:04:58.434762 15230 rpcz_store.cc:276] 0709 13:04:56.719893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.783679 (+ 63786us) service_pool.cc:224] Handling call
0709 13:04:58.429686 (+1646007us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54003d90d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:58.429840 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185922432}
I20260709 13:04:58.408247 20919 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:04:58.440266 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:58.429656 20791 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.418110 20957 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.411983 20423 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.437868 20717 consensus_queue.cc:1243] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:58.420732 20936 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.446076 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:58.454557 15082 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:58.405087 20956 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.439982 20883 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:58.440639 20964 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:58.444693 20907 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:58.463867 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 94968.4us
W20260709 13:04:58.464360 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.464632 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:04:58.463596 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 159656us
I20260709 13:04:58.496042 15085 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:58.496160 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.498543 15082 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:58.499449 15233 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:58.522267 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.519382 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:58.541569 15084 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:04:58.543067 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.459444 20604 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:58.572566 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50389.4us
W20260709 13:04:58.570861 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 332775us
I20260709 13:04:58.573640 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.579710 20835 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:58.580971 20835 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:58.581384 20835 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:58.582288 20835 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.583350 20936 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.584255 20907 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.583475 20835 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:58.590040 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:58.591044 15156 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 11: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:58.540251 20386 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.520701 20612 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.597270 20646 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:58.600530 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.602280 20646 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:58.602666 20646 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:58.602727 20423 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.573432 20604 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.603823 20646 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.605348 20604 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:58.605640 20646 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:58.607559 20965 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:04:58.617857 20781 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:04:58.620046 20963 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:04:58.602747 20870 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:04:58.623399 20962 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:04:58.591876 20916 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:58.629364 20916 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.631832 20609 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.457154 20920 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.631959 20916 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:58.645818 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:58.636696 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 424943us
W20260709 13:04:58.639757 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.648239 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47794.4us
I20260709 13:04:58.697118 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.697422 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.651211 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:58.699863 15848 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:58.708493 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.710142 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:58.713308 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:04:58.714629 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63519.2us
W20260709 13:04:58.691527 20954 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.002s
I20260709 13:04:58.600965 20871 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:58.727680 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.728615 15079 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:58.692468 20959 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.741788 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 107959us
I20260709 13:04:58.740747 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:58.742403 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 210998us
I20260709 13:04:58.594183 20804 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:58.705013 20966 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:58.698738 20604 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:58.739099 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.755306 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:58.728497 20612 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:58.770287 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 665097us
W20260709 13:04:58.770509 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 62137.9us
I20260709 13:04:58.734663 20790 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.771597 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.771770 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.772344 15233 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:58.773072 20804 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.774358 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.775100 15235 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:58.704883 20964 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:58.777400 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.785231 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.786372 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.787132 15233 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.789218 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.740696 20980 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:58.795847 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:58.798418 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:58.761112 20797 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:58.800276 15079 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:58.805154 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:58.805711 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.805996 15231 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.765967 20612 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.806825 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:04:58.805212 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54006.6us
I20260709 13:04:58.807492 15229 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:58.809734 15234 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:58.810925 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:04:58.807819 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.741524 20722 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:04:58.748658 20847 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:04:58.804514 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 113700us
I20260709 13:04:58.800033 20936 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.801043 20957 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:58.813649 15081 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:58.742439 20954 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.793859 20604 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:58.812251 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:58.801414 20609 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:58.758230 20959 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.803552 20934 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:58.807206 20797 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.823941 15235 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:58.808279 20907 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.811705 20983 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:04:58.888712 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82516.9us
I20260709 13:04:58.820065 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:04:58.837637 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.808640 20423 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.898002 15086 raft_consensus.cc:2368] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:58.899473 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:58.811169 20612 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.843894 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:58.902336 15086 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:58.910362 20932 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:58.911221 20932 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:58.913630 20423 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.912170 20612 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:58.914212 20932 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.915848 20612 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:58.837114 20790 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.946673 20612 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:58.822575 20970 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.882174 20797 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:59.022233 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:58.841406 20934 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.021260 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.022915 15081 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
W20260709 13:04:59.018026 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 219673us
I20260709 13:04:59.032785 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.033497 15233 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:04:59.034397 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 279122us
I20260709 13:04:59.036250 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.036909 15231 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:59.014889 20976 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:59.040010 15235 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:58.833379 20936 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.041142 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.048990 15231 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:04:59.049551 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1304) took 1211 ms. Trace:
I20260709 13:04:59.051692 15161 rpcz_store.cc:276] 0709 13:04:57.837990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.913007 (+ 75017us) service_pool.cc:224] Handling call
0709 13:04:57.913302 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.940519 (+ 27217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.049299 (+1108780us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5400187ed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:04:59.049491 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59629312}
I20260709 13:04:59.059011 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.060293 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.062345 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:58.837924 20871 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.066917 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:04:59.063565 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.068051 15847 raft_consensus.cc:2368] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:59.064293 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.028532 20934 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:59.066054 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.068815 15229 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:59.070668 15235 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:04:59.071569 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.037214 20974 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.056595 20795 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.005s
I20260709 13:04:59.018131 20790 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.027940 20785 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:04:58.917229 20983 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.065343 20976 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:59.084314 15235 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:04:59.076375 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 93165us
I20260709 13:04:59.094209 15233 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:59.026003 20970 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.100091 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:59.101267 15233 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:59.122327 15229 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:04:59.078066 20932 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:04:59.131625 20932 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:59.132031 20932 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:59.137784 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2937) took 1046 ms. Trace:
I20260709 13:04:59.138258 15158 rpcz_store.cc:276] 0709 13:04:58.090960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.135615 (+ 44655us) service_pool.cc:224] Handling call
0709 13:04:59.137572 (+1001957us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b5400378554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:59.137697 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121341696}
I20260709 13:04:59.149865 20604 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:59.151079 20870 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:59.150987 20604 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.153180 20936 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.089699 20423 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:59.086971 20916 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:04:59.156728 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:04:59.157063 20777 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.157660 20777 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.127805 15233 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.160409 15233 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:59.161198 20954 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.158077 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:59.162315 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2934) took 1076 ms. Trace:
I20260709 13:04:59.162720 15154 rpcz_store.cc:276] 0709 13:04:58.085595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.141594 (+ 55999us) service_pool.cc:224] Handling call
0709 13:04:59.162127 (+1020533us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b540037ee54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:04:59.162257 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52880768}
I20260709 13:04:59.164100 20790 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.164394 20932 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:59.167845 20835 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.000s
I20260709 13:04:59.168743 20976 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.170751 15231 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:59.171784 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.151512 20870 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:59.171797 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.177999 15230 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:04:59.173324 15233 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:04:59.179646 15233 pending_rounds.cc:85] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 10
W20260709 13:04:59.179584 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 80768.3us
I20260709 13:04:59.180223 15233 pending_rounds.cc:107] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:04:59.180931 15233 raft_consensus.cc:2889] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:59.081436 20974 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.180691 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:59.182478 15087 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
W20260709 13:04:59.182931 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 422742us
I20260709 13:04:59.174659 15128 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:04:59.182886 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:59.172772 20989 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:59.172783 20998 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:59.195531 21004 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:59.196593 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.196764 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:59.246465 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49794us
I20260709 13:04:59.247395 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:59.197016 21002 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.198719 21009 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:59.199415 20580 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:59.200001 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.254014 15201 leader_election.cc:400] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:59.254509 15201 leader_election.cc:403] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:04:59.256201 15202 leader_election.cc:400] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 10 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:59.201915 15231 pending_rounds.cc:85] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 5
I20260709 13:04:59.258015 15231 pending_rounds.cc:107] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:04:59.258456 15231 raft_consensus.cc:2889] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:59.202468 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:59.204138 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.205713 20847 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:59.177671 20777 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.218032 20970 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.219306 15235 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:04:59.220328 20984 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
W20260709 13:04:59.221355 20993 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:04:59.278198 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:59.221879 20646 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:59.225284 20955 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.000s
W20260709 13:04:59.184417 20969 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:04:59.184301 20994 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.167s
I20260709 13:04:59.231086 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1447) took 1194 ms. Trace:
I20260709 13:04:59.281086 21003 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:59.185683 20981 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:59.233583 20791 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.000s
I20260709 13:04:59.185506 20982 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:59.185750 20972 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:04:59.192131 20846 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:59.385440 20846 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:59.386648 20846 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.181130 20954 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:04:59.189777 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:04:59.249418 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.181484 20966 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:59.277489 15154 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:04:59.244485 20923 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.807s	user 0.007s	sys 0.005s
I20260709 13:04:59.330894 15842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:59.267350 20894 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:59.254006 20722 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:59.244580 20998 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:04:59.355113 15839 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:59.275871 21009 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:04:59.270769 20987 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11.
I20260709 13:04:59.187077 20907 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:04:59.368762 15235 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:59.355655 20955 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.280375 21004 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:04:59.185431 20968 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
I20260709 13:04:59.372753 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:04:59.370476 15845 rpcz_store.cc:276] 0709 13:04:58.037114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.038023 (+   909us) service_pool.cc:224] Handling call
0709 13:04:58.038475 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.038914 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:58.039273 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.272186 (+232913us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.450707 (+178521us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b180042de40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:04:58.606809 (+156102us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:59.164428 (+557619us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b2c000fb978. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:04:59.164800 (+   372us) log.cc:844] Serialized 26 byte log entry
0709 13:04:59.173044 (+  8244us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:59.173086 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.173151 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.230823 (+ 57672us) raft_consensus.cc:1685] finished
0709 13:04:59.230850 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.231013 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24213,"prepare.queue_time_us":536,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":54861,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":2587,"spinlock_wait_cycles":185813248,"thread_start_us":231711,"threads_started":1,"wal-append.queue_time_us":401327}
I20260709 13:04:59.380363 21008 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:59.381457 20612 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:04:59.382130 20791 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.378476 20982 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:04:59.393813 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.012s	sys 0.022s
W20260709 13:04:59.469669 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.012s	sys 0.022s
W20260709 13:04:59.470270 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 681083us
I20260709 13:04:59.471091 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:59.267948 21002 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:59.395601 15054 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.631s (SENT) [suppressed 22 similar messages]
W20260709 13:04:59.451793 20993 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:04:59.399322 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 299284us
I20260709 13:04:59.408564 20923 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.372184 20804 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.403885 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1450) took 1246 ms. Trace:
I20260709 13:04:59.485011 15840 rpcz_store.cc:276] 0709 13:04:58.157213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.157323 (+   110us) service_pool.cc:224] Handling call
0709 13:04:58.157658 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.157974 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.157999 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.159456 (+  1457us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.419874 (+260418us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:58.419914 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.419986 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.403637 (+983651us) raft_consensus.cc:1685] finished
0709 13:04:59.403662 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.403827 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":556,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":260099,"threads_started":1,"wal-append.queue_time_us":578012}
I20260709 13:04:59.404516 15843 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:59.375002 20959 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.484707 20804 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.491755 20804 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:04:59.282831 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:04:59.442188 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 384323us
I20260709 13:04:59.492328 20982 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.354549 20870 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.494572 20870 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:59.418656 15841 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:59.401223 20646 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:59.398772 15842 raft_consensus.cc:606] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:59.497658 20646 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.498206 20646 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:59.480226 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 84695.8us
I20260709 13:04:59.480369 21009 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.502638 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:04:59.503357 15088 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:04:59.503494 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:59.504086 15081 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:59.505035 15085 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:59.506057 15232 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:04:59.479140 15155 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:59.497716 15842 raft_consensus.cc:614] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:59.447104 20954 raft_consensus.cc:2764] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:59.493587 20982 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:59.493527 15127 leader_election.cc:400] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
W20260709 13:04:59.496614 20958 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.046s	user 0.006s	sys 0.003s
I20260709 13:04:59.520906 15127 leader_election.cc:403] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:04:59.512956 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 230242us
W20260709 13:04:59.512317 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 322616us
I20260709 13:04:59.536156 15236 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:04:59.537918 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56130.5us
I20260709 13:04:59.539484 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:59.537976 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:04:59.535385 20968 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:59.552237 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1241) took 1187 ms. Trace:
I20260709 13:04:59.550498 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:59.555894 15228 rpcz_store.cc:276] 0709 13:04:58.365389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.365640 (+   251us) service_pool.cc:224] Handling call
0709 13:04:58.366080 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.648276 (+282196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.648315 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:58.649473 (+  1158us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.809137 (+159664us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.833449 (+ 24312us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.149332 (+315883us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b180037dec0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:04:59.238013 (+ 88681us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.238346 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.238452 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.550699 (+312247us) raft_consensus.cc:1685] finished
0709 13:04:59.550733 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.552163 (+  1430us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38770,"prepare.queue_time_us":61700,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":12274,"raft.queue_time_us":392720,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":29992,"spinlock_wait_cycles":60142336,"thread_start_us":327815,"threads_started":2,"wal-append.queue_time_us":107023}
I20260709 13:04:59.559309 15228 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:59.469862 20612 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.520231 20894 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:59.534978 20981 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.571858 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.547392 20847 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:59.549944 20846 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.590297 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 50044.5us
W20260709 13:04:59.590910 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51499.4us
I20260709 13:04:59.593063 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:04:59.591703 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 98386.8us
I20260709 13:04:59.522250 20797 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.568537 20386 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:59.469429 21008 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.600114 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.602835 15161 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:04:59.593036 20847 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:04:59.605350 21007 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:04:59.605867 20791 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.583289 20987 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.609926 15842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:04:59.610541 15842 raft_consensus.cc:606] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:04:59.610610 20797 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.565133 20954 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:59.602208 20386 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.612650 20386 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:59.563813 20998 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.610947 15842 raft_consensus.cc:614] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:59.583636 21004 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.584724 20981 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.617895 21002 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.619405 20923 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.601028 21016 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:04:59.620590 21018 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:04:59.621227 21014 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:59.621802 20955 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.622632 21018 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:59.621209 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:59.627085 20934 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.001s
I20260709 13:04:59.627583 20934 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.607209 21006 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:04:59.629820 21006 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.632094 21006 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.632824 20987 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.635124 20987 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.638727 21025 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:59.591291 20974 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.581012 20871 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:59.624635 21008 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.646806 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:04:59.662480 21017 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:59.670030 20612 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.597277 20846 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.682348 20846 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:04:59.645782 20847 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:04:59.646262 20907 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:59.619422 20981 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.647043 20923 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:04:59.712939 20907 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:59.728065 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 132246us
I20260709 13:04:59.734581 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.735383 15157 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:04:59.737147 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:59.737826 15085 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:04:59.739054 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.739696 15237 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:04:59.740820 15237 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:59.741442 15231 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:59.713709 20934 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:04:59.751972 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:59.745144 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:04:59.784024 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.725792 21027 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.785450 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.785470 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.786326 15159 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:04:59.783716 15085 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:04:59.792450 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:59.792272 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:59.799286 15081 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:04:59.806936 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63972.1us
I20260709 13:04:59.784582 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:59.808094 15161 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:04:59.818039 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:59.819082 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:04:59.822834 15231 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:59.823859 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.824436 15237 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:59.825608 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 263324us
I20260709 13:04:59.829592 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:04:59.830219 15231 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:04:59.827495 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:59.832391 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:59.787637 21030 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:04:59.806607 21025 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:04:59.828421 21019 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:04:59.839808 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:04:59.828013 20981 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:59.840801 20846 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:59.842554 15237 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:04:59.845304 21040 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.839524 20870 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:04:59.838119 20847 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:04:59.805663 21027 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.829258 20987 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:59.855559 21030 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.858101 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:04:59.862510 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:04:59.855193 21039 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:59.879125 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:04:59.832036 20386 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:04:59.881114 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.881973 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:59.873147 20846 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:59.884977 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 288034us
I20260709 13:04:59.852880 20646 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.891505 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:04:59.878191 20834 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.878611 20894 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:59.883453 20981 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:59.881031 21039 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:04:59.884488 20987 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:59.901437 15161 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:59.900578 20981 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.900682 20834 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.855329 20907 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.909811 15160 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:04:59.890044 21002 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:59.888804 20846 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:59.899132 21025 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:04:59.912091 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:04:59.914592 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.875793 21040 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:04:59.932516 15157 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:04:59.897902 21021 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:59.938650 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.899152 20894 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.971582 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:59.971802 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 72048.1us
I20260709 13:04:59.972422 15159 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:59.972445 15161 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:04:59.975908 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:04:59.913453 21029 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:04:59.895876 21008 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:04:59.910238 20981 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:59.976507 15161 raft_consensus.cc:2376] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16. Re-sending same reply.
I20260709 13:04:59.981564 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:04:59.936599 21002 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:04:59.898171 21036 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 13:04:59.974423 20894 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:00.106974 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 279542us
I20260709 13:04:59.976306 21021 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:00.103437 20982 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:00.117615 21036 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.103127 21038 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:00.150226 21038 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.150413 15842 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:00.150672 21038 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:00.152858 21037 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:05:00.154217 21041 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:05:00.160989 20847 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.161788 20847 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.164321 20847 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.177692 21002 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.181989 21030 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.186451 21027 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.191035 20834 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.194183 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:00.195240 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:00.145854 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1381) took 1449 ms (client timeout 1760 ms). Trace:
W20260709 13:05:00.199805 15087 rpcz_store.cc:269] 0709 13:04:58.696619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.696758 (+   139us) service_pool.cc:224] Handling call
0709 13:05:00.145676 (+1448918us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540011add4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:00.145796 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864020224}
I20260709 13:05:00.147034 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1310) took 1907 ms. Trace:
I20260709 13:05:00.195299 15842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:00.202943 15842 raft_consensus.cc:606] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:00.208046 20960 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:00.208529 20960 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:00.209383 20960 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.210623 20960 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:00.212502 21044 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:00.136446 20955 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:00.214502 20955 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.201028 15155 rpcz_store.cc:276] 0709 13:04:58.239948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.278867 (+ 38919us) service_pool.cc:224] Handling call
0709 13:04:58.279152 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.506715 (+1227563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.146796 (+640081us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540017eb54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:00.146973 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2759298432}
I20260709 13:05:00.134729 20987 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.217553 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.279772 21021 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:00.232666 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 259703us
I20260709 13:05:00.353087 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:00.347698 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 239790us
I20260709 13:05:00.261431 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1438) took 2053 ms. Trace:
I20260709 13:05:00.365100 15839 rpcz_store.cc:276] 0709 13:04:58.207746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.274103 (+ 66357us) service_pool.cc:224] Handling call
0709 13:04:58.274465 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.414157 (+1139692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.261178 (+847021us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:00.261370 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484725120}
W20260709 13:05:00.353744 20986 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.495s	user 0.006s	sys 0.002s
I20260709 13:05:00.366884 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.367587 15842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:00.367820 15230 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:00.367542 15162 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:00.368237 15842 raft_consensus.cc:606] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:00.368625 15842 raft_consensus.cc:614] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:00.369421 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:00.369722 15230 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:00.369989 15162 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:00.371873 15230 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:00.375363 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.376129 15236 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:00.307507 20787 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.135568 21039 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.378199 15163 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:05:00.251693 20906 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:00.260946 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:00.237389 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.352413 20955 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.397388 15236 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:05:00.401435 21043 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:00.403715 15155 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:00.276530 20791 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:05:00.225991 21018 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.169s
I20260709 13:05:00.148643 20982 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.420782 15163 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:05:00.393005 21039 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:00.379621 20787 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:00.416559 20988 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.394s	user 0.000s	sys 0.010s
I20260709 13:05:00.226116 21036 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.425760 21040 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.399909 20906 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.472473 15155 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:00.470105 21048 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.486248 15162 raft_consensus.cc:2410] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:05:00.498731 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 136111us
I20260709 13:05:00.500118 15155 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:05:00.502010 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 137899us
I20260709 13:05:00.503460 15084 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:00.503362 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:00.506001 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:00.506078 20956 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.506603 15088 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:00.506784 15082 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:00.506177 15079 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:05:00.508625 15079 pending_rounds.cc:85] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 8
I20260709 13:05:00.508920 15079 pending_rounds.cc:107] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:05:00.509160 15079 raft_consensus.cc:2889] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.482285 21018 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:00.515688 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:00.522529 21023 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.822s	user 0.004s	sys 0.008s
I20260709 13:05:00.526881 20646 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:00.531718 20920 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:00.517481 21045 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:05:00.536410 20906 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.511164 20894 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.539064 20894 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.539584 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1447) took 1261 ms. Trace:
I20260709 13:05:00.540000 15846 rpcz_store.cc:276] 0709 13:04:59.277798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.277931 (+   133us) service_pool.cc:224] Handling call
0709 13:05:00.539383 (+1261452us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540041cdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:00.539523 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716284672}
I20260709 13:05:00.543431 21008 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:00.544543 21008 consensus_queue.cc:1243] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:00.491797 21048 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.506749 20956 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.548002 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:00.553028 15163 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:00.550438 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1448) took 1198 ms. Trace:
I20260709 13:05:00.559044 15841 rpcz_store.cc:276] 0709 13:04:59.351731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.353262 (+  1531us) service_pool.cc:224] Handling call
0709 13:04:59.354976 (+  1714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.497131 (+142155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.550134 (+1053003us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b54005a3a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:00.550364 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145985920}
I20260709 13:05:00.551626 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:00.540093 20981 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:00.544802 20846 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.001s
I20260709 13:05:00.552579 21046 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.592020 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1249) took 1551 ms. Trace:
I20260709 13:05:00.598905 15234 rpcz_store.cc:276] 0709 13:04:59.040754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.040885 (+   131us) service_pool.cc:224] Handling call
0709 13:05:00.591817 (+1550932us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54002779d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:00.591953 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522641152}
I20260709 13:05:00.592022 15083 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:00.600584 15083 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:00.601814 15083 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.546757 21048 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.569070 21055 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.545130 21008 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:00.612300 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.613076 15155 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:00.603861 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.615798 15163 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:00.619114 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:00.619863 15236 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.619856 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.573642 21045 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:05:00.626910 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.609046 20646 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:00.614634 21008 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:00.618160 20604 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:00.627537 15230 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:00.611802 20982 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.632016 15814 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:00.604400 20846 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.636941 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:00.640573 15080 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:00.619148 21048 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:00.622174 21057 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:00.637909 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 130108us
I20260709 13:05:00.638276 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:00.607728 20907 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:05:00.609301 21021 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:00.617645 21055 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.634438 20906 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.626754 21040 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:05:00.645740 20834 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:00.641355 20787 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.634927 20975 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:05:00.681550 15236 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:00.636982 20646 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.685834 21036 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:00.687354 21052 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:05:00.690200 21038 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.002s
I20260709 13:05:00.690825 21038 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.673252 21008 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.694073 21008 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:00.697829 15084 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:00.698676 21060 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:05:00.700008 21060 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:00.700456 21060 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:00.700770 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1253) took 1449 ms. Trace:
I20260709 13:05:00.701552 21060 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.702387 20956 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.676928 20834 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.705718 20834 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:00.706815 21055 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.711076 21003 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:00.711633 21003 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.712092 21038 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:00.713519 20981 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.714882 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.665678 20870 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:00.715765 20870 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:00.716265 20870 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:00.720433 21003 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.721356 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:00.721973 15157 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:00.722018 20846 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:00.722734 21003 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.724205 20983 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:00.728375 20386 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:00.729688 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:00.732146 20995 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:00.732642 20995 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:00.733713 21021 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:00.734194 21021 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:00.734602 21021 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.734758 20532 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:00.736414 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.660364 21057 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.741034 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1320) took 1143 ms. Trace:
I20260709 13:05:00.741494 15158 rpcz_store.cc:276] 0709 13:04:59.597687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.597818 (+   131us) service_pool.cc:224] Handling call
0709 13:04:59.598119 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.739201 (+1141082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.739236 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:00.739277 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:00.739314 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.740756 (+  1442us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540033ed54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:00.740805 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.740970 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2623760640}
I20260709 13:05:00.746133 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:00.656541 21048 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.747669 15162 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.678732 20906 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.749424 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2222) took 1963 ms. Trace:
I20260709 13:05:00.749861 15232 rpcz_store.cc:276] 0709 13:04:58.786316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.792936 (+  6620us) service_pool.cc:224] Handling call
0709 13:05:00.749215 (+1956279us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:00.749364 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1596680576}
I20260709 13:05:00.751433 15235 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:00.753249 15158 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:00.701342 15228 rpcz_store.cc:276] 0709 13:04:59.251721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.560185 (+308464us) service_pool.cc:224] Handling call
0709 13:04:59.560539 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.569213 (+  8674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.569254 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.569868 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:05:00.537803 (+967935us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b180037df80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:00.617750 (+ 79947us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:00.617808 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.617899 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:00.700353 (+ 82454us) raft_consensus.cc:1685] finished
0709 13:05:00.700383 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.700686 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11126,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8551,"spinlock_wait_cycles":4046208,"thread_start_us":79239,"threads_started":1,"wal-append.queue_time_us":1088202}
W20260709 13:05:00.759101 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.763459 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1443) took 1721 ms. Trace:
I20260709 13:05:00.763993 15843 rpcz_store.cc:276] 0709 13:04:59.042422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.042521 (+    99us) service_pool.cc:224] Handling call
0709 13:04:59.404338 (+361817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.487222 (+ 82884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.763172 (+1275950us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54006dc250. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:00.763366 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831395584}
W20260709 13:05:00.767127 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.769012 15228 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:00.769289 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:00.758214 15162 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:05:00.772913 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.777720 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:00.781845 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:00.788529 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:00.791837 15846 raft_consensus.cc:2410] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:00.792578 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:00.803397 15841 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:00.736685 20532 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:00.734885 20609 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:05:00.780632 20981 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.797705 21060 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.788165 20983 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:00.733765 20995 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.838114 21057 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.841794 21048 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.671943 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:00.732525 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:00.895707 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 223930us
I20260709 13:05:00.903126 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:00.916968 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:00.787076 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.930581 15236 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:00.922165 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:00.923956 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:00.925767 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:00.937332 15228 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:00.945382 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 139409us
I20260709 13:05:00.926290 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:00.948278 20386 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.948350 15162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34534
I20260709 13:05:00.949470 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:00.949744 15162 raft_consensus.cc:606] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:00.950548 20532 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:00.949843 20386 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:00.951596 20781 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:00.953403 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:00.955937 20646 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.944463 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:00.959081 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:00.926918 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:00.971395 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 210666us
I20260709 13:05:00.972431 20995 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:00.854745 21021 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:00.976248 21039 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:00.978816 20906 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:00.979472 21021 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:00.987920 21078 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:00.988495 21038 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:00.988981 15232 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:00.990202 21038 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:00.990687 21038 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:00.997159 20983 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:00.980899 15810 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:01.005620 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 272825us
I20260709 13:05:00.966586 20532 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.978861 21045 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:01.007604 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:01.008247 21040 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:01.007513 20532 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.989130 21078 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.992334 21038 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.013947 21038 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:01.015255 21078 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.019398 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.020478 15161 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:00.976558 21044 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:01.021349 21070 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:01.030076 21070 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:01.009423 20981 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.033861 20981 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:00.998176 20983 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.040199 20983 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.045246 20846 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.045778 15083 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:01.024288 21044 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:01.026332 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 82107.4us
I20260709 13:05:01.026582 20781 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:01.027341 21066 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.070442 21066 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.052428 20975 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:05:01.073482 20975 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:01.073945 20975 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:01.075150 20975 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.027992 21047 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.028574 21021 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:01.093678 21021 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.096259 21021 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.064913 21044 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.101016 20981 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.031926 21070 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.102593 21078 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:01.107734 20781 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:01.108659 20781 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.049839 20997 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.111994 20997 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.113155 20781 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.114693 20997 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.008584 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.053685 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2512) took 1108 ms. Trace:
W20260709 13:05:01.067126 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.069304 20985 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:01.121166 20985 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:01.122041 15052 connection.cc:256] RPC call timeout handler was delayed by -1.0833371549999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:05:01.007844 21045 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.125958 21045 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:01.081672 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 101094us
I20260709 13:05:01.128535 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.120496 15161 rpcz_store.cc:276] 0709 13:04:59.915210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.018672 (+1103462us) service_pool.cc:224] Handling call
0709 13:05:01.023338 (+  4666us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:01.130241 20894 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.131035 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.132906 21008 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:01.133563 20981 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.134405 20981 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.135286 21008 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:01.121201 21064 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.009764 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.137831 20386 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:01.137701 21064 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:01.138567 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 129032us
I20260709 13:05:01.138933 20386 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.140180 21030 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.123112 20985 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.135668 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.124305 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:01.091392 21047 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.119714 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.144917 15155 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:01.150916 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 190831us
I20260709 13:05:01.151816 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.142100 20646 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.164467 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.164595 20997 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:01.164930 20646 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.167423 20646 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.142033 20936 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:01.168190 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.168385 20936 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.170904 20936 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.171011 21018 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:01.143733 20919 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:01.171651 21018 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:01.172030 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:01.173825 21025 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:01.177874 15228 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:01.172220 20919 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.150009 20835 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:01.147805 20985 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:01.150530 20960 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.939s	user 0.003s	sys 0.021s
I20260709 13:05:01.222189 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.223235 15087 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
W20260709 13:05:01.225097 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.140810 21064 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.175695 20997 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:05:01.187134 20532 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.189184 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.189818 20981 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:01.193879 20920 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.661s	user 0.014s	sys 0.010s
I20260709 13:05:01.197355 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.199723 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.201987 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.202888 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.204416 15082 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:01.205322 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.206120 20998 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:01.240132 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56183.5us
I20260709 13:05:01.206743 21022 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:01.257035 21022 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.257277 20975 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.257527 21022 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.264770 20997 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.265549 20997 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.266723 20934 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.267289 20934 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.268525 15156 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:01.207283 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.149924 21030 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:05:01.208392 21053 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:01.291770 21053 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:01.292914 21053 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.294438 21053 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.210960 20386 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.215046 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1318) took 1678 ms. Trace:
I20260709 13:05:01.297164 15154 rpcz_store.cc:276] 0709 13:04:59.536493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.536637 (+   144us) service_pool.cc:224] Handling call
0709 13:04:59.553923 (+ 17286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.554257 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.554282 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.554773 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.213854 (+1659081us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b180042d9c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1
0709 13:05:01.214248 (+   394us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:01.214336 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.214459 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.214513 (+    54us) raft_consensus.cc:1685] finished
0709 13:05:01.214524 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.214830 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89611,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8268,"spinlock_wait_cycles":70128000,"thread_start_us":62293,"threads_started":1,"wal-append.queue_time_us":144107,"wal-append.run_cpu_time_us":20533,"wal-append.run_wall_time_us":1411029}
I20260709 13:05:01.147406 21047 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.141175 21066 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.235582 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.301070 20532 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.305882 15163 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:01.305061 21022 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:01.308977 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68135.1us
I20260709 13:05:01.310312 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:01.311793 21060 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:01.312184 21060 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.312507 21060 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.103523 21078 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.243916 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:01.245098 21018 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.249691 15085 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:01.256008 15842 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:01.207531 20835 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:01.267935 20997 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:01.273700 20846 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.178964 21045 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.274051 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:01.274307 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 125688us
I20260709 13:05:01.269809 15155 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:01.275097 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:01.277062 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.277293 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:01.277916 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:01.279882 15847 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:01.277839 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.288983 20982 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.230366 20870 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.009s
I20260709 13:05:01.230577 20646 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.351840 15160 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:01.352162 21045 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.351924 20646 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.321940 21018 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:01.323582 20835 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:01.324900 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.325644 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.330178 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:01.320632 21078 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.371676 15842 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:05:01.350889 20870 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.351207 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:01.318364 20998 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.348009 15845 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:05:01.349798 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.442248 15847 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:01.494306 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.404649 15839 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:01.433736 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:01.462707 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 292207us
I20260709 13:05:01.500654 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.509498 15155 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:01.375635 15846 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:05:01.452479 21076 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:05:01.515331 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:01.513052 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100767us
I20260709 13:05:01.517159 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:01.520931 20834 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.003s
I20260709 13:05:01.478811 21022 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.376952 21045 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:01.497097 20919 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.536139 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.539278 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.540714 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.541594 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:01.542021 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:01.542526 15157 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.014s
I20260709 13:05:01.542810 15845 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.543012 15157 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:01.544274 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.545619 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2232) took 1697 ms. Trace:
I20260709 13:05:01.522104 20646 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.548112 20932 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:01.533330 20604 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:01.551020 20604 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.531560 20894 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:01.555032 21044 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.557159 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.557619 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:01.558287 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2508) took 2386 ms. Trace:
I20260709 13:05:01.558802 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.559562 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:01.560520 15160 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:01.562024 20791 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.561391 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.558691 15235 rpcz_store.cc:276] 0709 13:04:59.171743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.462375 (+290632us) service_pool.cc:224] Handling call
0709 13:05:00.751249 (+1288874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.766138 (+ 14889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.557911 (+791773us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002d4450. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:01.558088 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862676864}
I20260709 13:05:01.560964 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.563345 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:01.563493 20870 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:01.570186 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.562662 15161 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:01.564054 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:01.570119 15843 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:01.566006 15080 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:01.586743 15158 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:01.561738 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.537421 20834 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.568612 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:01.578398 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 100242us
W20260709 13:05:01.533946 21038 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.003s
I20260709 13:05:01.375123 21078 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 13:05:01.579524 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.562050 15163 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:01.584194 15848 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:01.581638 15155 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:01.587986 15842 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:01.595325 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.595768 15054 connection.cc:256] RPC call timeout handler was delayed by -1.1777963499999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 13:05:01.511983 20998 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.387951 20846 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:01.614465 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 89799.7us
I20260709 13:05:01.405745 20982 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.619144 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.557648 20934 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.625077 15228 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 11.
I20260709 13:05:01.626699 20604 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:01.626909 15843 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:01.627317 20791 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:01.630117 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 189060us
I20260709 13:05:01.631803 15232 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:01.633526 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:01.546146 15231 rpcz_store.cc:276] 0709 13:04:59.833710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.833867 (+   157us) service_pool.cc:224] Handling call
0709 13:04:59.845219 (+ 11352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.845589 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.845616 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.155509 (+1309893us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b2c00032e28. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:05:01.173548 (+ 18039us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.314665 (+141117us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:01.318706 (+  4041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.318838 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.497559 (+178721us) raft_consensus.cc:1685] finished
0709 13:05:01.497581 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.530753 (+ 33172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7742,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"spinlock_wait_cycles":26101120,"thread_start_us":140626,"threads_started":1,"wal-append.queue_time_us":271361}
I20260709 13:05:01.621590 15085 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:01.628625 15843 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:01.639559 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.641028 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.590051 21045 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:01.643678 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49469.9us
I20260709 13:05:01.644524 15844 raft_consensus.cc:2368] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
I20260709 13:05:01.589465 21064 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.630261 20791 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:01.660918 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.668545 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.669948 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.671124 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.672515 15079 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:01.673574 15233 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:01.675348 21045 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.680966 15234 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:01.684880 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.685539 15085 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:01.690547 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149090us
I20260709 13:05:01.653781 20983 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.702036 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.642235 20975 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:05:01.686679 15162 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:01.706694 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.718235 15083 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:01.708498 15081 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:01.764928 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.765609 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.754750 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:01.766259 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.856173 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:01.842639 21060 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.004s
I20260709 13:05:01.856931 21080 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:01.857517 21080 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.857129 21060 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.857920 21080 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:01.859722 21060 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.852861 15236 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:05:01.866556 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 98385.2us
I20260709 13:05:01.875388 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:01.766655 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 71120.3us
I20260709 13:05:01.777509 15079 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:01.714949 20791 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.864687 15847 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:01.843026 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.889648 20976 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.764959 21082 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.892513 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.893591 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.895380 21044 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.895668 21078 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:01.790861 20983 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:01.917120 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 95471.8us
I20260709 13:05:01.917960 21082 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.919076 20983 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:01.919509 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.920073 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:01.920779 15079 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:01.921358 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1329) took 1091 ms. Trace:
I20260709 13:05:01.921732 15159 rpcz_store.cc:276] 0709 13:05:00.548677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.738398 (+189721us) service_pool.cc:224] Handling call
0709 13:05:00.755932 (+ 17534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.756392 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.756433 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.759738 (+  3305us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.187239 (+427501us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:01.187342 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.187426 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.632426 (+445000us) raft_consensus.cc:1685] finished
0709 13:05:01.632444 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.640495 (+  8051us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1549,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7929,"thread_start_us":414363,"threads_started":1,"wal-append.queue_time_us":560271}
I20260709 13:05:01.923112 15079 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:01.927301 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58971.1us
I20260709 13:05:01.911770 20934 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.927273 15083 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:05:01.928830 15083 pending_rounds.cc:85] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:05:01.928591 20934 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.929195 15083 pending_rounds.cc:107] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:05:01.929522 15083 raft_consensus.cc:2889] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:01.931034 15083 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:01.931197 20934 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.932916 21082 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:01.917228 21078 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.936271 21078 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.940603 21044 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.900599 15232 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:01.901798 21022 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:01.912286 15841 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:01.917248 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:01.861122 20981 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.841014 21053 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.005s
I20260709 13:05:01.945885 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:01.953889 21022 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.954206 20791 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } }
I20260709 13:05:01.954903 20834 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.932934 20976 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:01.920970 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2297) took 1138 ms. Trace:
I20260709 13:05:01.950397 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:01.964313 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.976029 20982 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.977344 21022 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:01.978680 20976 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:01.979663 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.980077 20791 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:01.990238 20981 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:01.991364 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:01.971521 15234 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:01.971298 15084 rpcz_store.cc:276] 0709 13:05:00.504626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.504742 (+   116us) service_pool.cc:224] Handling call
0709 13:05:00.505009 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.698450 (+193441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.581635 (+883185us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54003cf354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:01.643132 (+ 61497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442708736}
I20260709 13:05:01.932717 15085 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:01.993634 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:01.999765 15088 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:02.013732 15236 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:02.014031 15087 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:02.010392 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.015718 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:02.021135 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.030220 15841 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:02.021116 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 50609.8us
I20260709 13:05:01.993335 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:02.022991 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55848.9us
I20260709 13:05:01.994217 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.032904 15846 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:02.011754 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:02.035085 15845 raft_consensus.cc:2410] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:05:02.036888 15085 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:02.028112 15080 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:02.014349 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:02.038883 15842 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 11: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:02.046249 15154 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:02.003408 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:02.047278 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.056452 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:02.059176 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:02.060746 15079 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:01.992813 20981 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:02.018417 20998 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.018329 21070 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.018484 21022 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.067999 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:02.068367 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.023422 21044 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.069024 15229 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:02.069350 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.069975 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.075198 15087 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:01.993871 20976 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:02.011479 20846 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.015759 21060 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:02.084928 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.053282 20997 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:02.089340 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:02.102977 15080 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:02.105181 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48836us
I20260709 13:05:02.105561 15087 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:02.106101 20791 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.106799 15848 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:02.106736 20791 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.107311 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:02.109030 20932 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
I20260709 13:05:02.112526 15160 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:02.113282 21042 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.054445 20985 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:02.113850 21042 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:02.114908 21018 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.001s
I20260709 13:05:02.114776 20985 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.063601 20870 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:02.116403 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:02.117183 15229 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:02.116068 20870 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:02.120538 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.137192 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47924.7us
I20260709 13:05:02.137969 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.146755 15088 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:02.150542 20976 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:02.150295 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:02.151827 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.152454 20976 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.152961 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:02.153084 20791 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:02.148388 20834 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:02.149923 15081 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:02.153091 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:02.146674 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1317) took 2935 ms (client timeout 1983 ms). Trace:
I20260709 13:05:02.150668 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:02.157482 15156 rpcz_store.cc:269] 0709 13:04:59.201790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.203931 (+  2141us) service_pool.cc:224] Handling call
0709 13:05:02.094233 (+2890302us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b540037f854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:02.137458 (+ 43225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4619538816}
I20260709 13:05:02.140031 20835 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:02.158047 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:02.173427 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:02.174794 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:02.174959 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.175714 15157 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:02.093497 20894 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:05:02.183641 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.184697 21044 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.184798 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.157240 20985 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.148063 21022 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.185176 20834 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:02.198279 20834 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.199410 21022 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.185498 15087 raft_consensus.cc:2376] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15. Re-sending same reply.
I20260709 13:05:02.101922 21070 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.202332 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.203778 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.204031 21070 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.193013 15159 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:02.196390 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.167940 20835 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.139734 21080 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.150780 21081 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:05:02.235289 15229 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:02.157274 21060 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:02.298026 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.221731 21098 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.302451 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1379) took 1026 ms. Trace:
I20260709 13:05:02.332036 15081 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
W20260709 13:05:02.235594 21008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.048s	sys 0.060s
I20260709 13:05:02.236338 20835 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:02.335731 15847 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:02.302966 15844 rpcz_store.cc:276] 0709 13:05:01.276398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.276626 (+   228us) service_pool.cc:224] Handling call
0709 13:05:02.240036 (+963410us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540074d454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:02.302371 (+ 62335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667592576}
I20260709 13:05:02.343612 20997 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:02.355373 15159 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:02.357955 21100 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:02.358675 21100 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.359104 21100 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:02.358024 20997 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:02.361528 20997 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.363830 20997 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.365054 21070 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:02.365662 21070 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.367916 21070 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.368453 21070 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:02.369181 20870 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:02.373908 15081 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:02.381336 21070 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.384820 21070 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.403195 21080 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:02.441438 21044 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.442425 21044 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:02.351537 21008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.049s	sys 0.060s
I20260709 13:05:02.457463 21060 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.472060 15086 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:05:02.459003 20604 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.010s	sys 0.034s
W20260709 13:05:02.485370 20875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.017s	sys 0.038s
W20260709 13:05:02.485857 20875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.017s	sys 0.038s
W20260709 13:05:02.501847 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.013s	sys 0.034s
W20260709 13:05:02.505434 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.013s	sys 0.035s
W20260709 13:05:02.508297 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 576418us
W20260709 13:05:02.529098 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.532015 21042 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.535178 20791 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:02.536207 20791 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.540199 20791 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:02.486835 20604 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.010s	sys 0.034s
W20260709 13:05:02.557881 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53021.1us
I20260709 13:05:02.561625 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.562400 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.562232 20998 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:05:02.562975 20916 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.041s
W20260709 13:05:02.570628 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 490111us
I20260709 13:05:02.572098 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2937) took 1351 ms. Trace:
I20260709 13:05:02.571218 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:02.574375 15086 rpcz_store.cc:276] 0709 13:05:01.200980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.202531 (+  1551us) service_pool.cc:224] Handling call
0709 13:05:02.552023 (+1349492us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b5400191754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:02.552185 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2023420800}
I20260709 13:05:02.574216 15154 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:02.581243 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.582257 15162 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:02.500679 21070 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:02.585116 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.548749 21044 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:02.605681 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:02.574172 20916 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.041s
I20260709 13:05:02.607609 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:02.605463 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:02.608796 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.605374 20934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.022s	sys 0.012s
I20260709 13:05:02.615945 15127 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:02.579802 21045 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:05:02.554315 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:02.606480 20791 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:02.615255 20934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.023s	sys 0.012s
I20260709 13:05:02.613868 20985 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.618816 21060 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:02.610742 21070 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:02.647250 15052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.003s	sys 0.026s
W20260709 13:05:02.647796 15052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.003s	sys 0.026s
W20260709 13:05:02.648151 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 540941us
W20260709 13:05:02.652189 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.637151 21060 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:02.649591 21053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.027s
I20260709 13:05:02.672226 20934 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:05:02.691438 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:02.648687 20975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.016s	sys 0.012s
W20260709 13:05:02.701047 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 62674.4us
I20260709 13:05:02.694195 20982 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:02.702670 20957 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:02.703271 20957 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:02.703410 20982 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:02.712741 20982 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.639526 20985 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:02.692802 21053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.028s
I20260709 13:05:02.718168 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:02.720671 15235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.018s	sys 0.054s
I20260709 13:05:02.673235 21060 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:02.723940 15157 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.006s	sys 0.012s
W20260709 13:05:02.725023 15157 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.006s	sys 0.012s
W20260709 13:05:02.715233 20975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.017s	sys 0.013s
W20260709 13:05:02.723394 15235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.018s	sys 0.054s
W20260709 13:05:02.715972 15234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.003s	sys 0.014s
W20260709 13:05:02.724402 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.031s	sys 0.093s
W20260709 13:05:02.730629 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.031s	sys 0.094s
W20260709 13:05:02.730681 15234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.014s
W20260709 13:05:02.731034 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 583683us
I20260709 13:05:02.738436 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:02.718065 20982 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:02.717242 21038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.010s	sys 0.025s
I20260709 13:05:02.696589 20934 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:05:02.693539 20791 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.751721 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:02.751678 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.008s	sys 0.016s
W20260709 13:05:02.755966 15228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.005s	sys 0.031s
W20260709 13:05:02.756503 15228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.005s	sys 0.031s
W20260709 13:05:02.751281 21038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.010s	sys 0.026s
I20260709 13:05:02.745705 21018 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:02.753501 15814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.013s	sys 0.009s
W20260709 13:05:02.834414 15814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.013s	sys 0.010s
W20260709 13:05:02.756295 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.016s
W20260709 13:05:02.756457 15083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.002s	sys 0.022s
W20260709 13:05:02.835263 15083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.002s	sys 0.022s
W20260709 13:05:02.814926 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63268.6us
I20260709 13:05:02.750839 21008 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:02.834074 15084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.013s	sys 0.033s
I20260709 13:05:02.839946 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:02.842655 15084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.013s	sys 0.033s
I20260709 13:05:02.843343 20935 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:02.845327 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.846040 20935 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:02.847548 20985 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.846256 15845 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
W20260709 13:05:02.850251 15085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.020s
W20260709 13:05:02.850615 15085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.003s	sys 0.020s
W20260709 13:05:02.858130 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 233620us
I20260709 13:05:02.833590 20870 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:02.868152 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 112431us
I20260709 13:05:02.753087 21081 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:05:02.765689 21111 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.836131 21108 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:02.841367 21008 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.898726 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.907915 15086 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:05:02.908422 15086 pending_rounds.cc:85] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 8
I20260709 13:05:02.908794 15086 pending_rounds.cc:107] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:02.909147 15086 raft_consensus.cc:2889] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:02.909533 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.909759 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:02.910279 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.910406 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.910779 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.911394 15843 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:02.911520 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.912281 15079 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:05:02.915750 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.916869 15842 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:02.918378 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:02.857501 21080 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.919256 15847 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
W20260709 13:05:03.028107 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.866801 20894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.024s
W20260709 13:05:03.044253 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 167707us
I20260709 13:05:02.832978 20982 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.044540 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:02.937114 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.015s	sys 0.040s
W20260709 13:05:03.048976 15232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.045s	sys 0.076s
W20260709 13:05:03.049986 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 895324us
W20260709 13:05:03.057085 15232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.045s	sys 0.076s
I20260709 13:05:03.055261 15081 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:03.061655 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:03.055450 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.015s	sys 0.040s
W20260709 13:05:03.066963 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 430179us
I20260709 13:05:03.068506 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.069813 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.071642 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.078269 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:02.870309 20935 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:03.068713 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.090812 15081 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:03.074802 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.098344 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.099359 15079 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:02.920742 20834 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.005s	sys 0.027s
I20260709 13:05:03.077550 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.106247 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:03.108748 21116 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.265s	user 0.000s	sys 0.000s
W20260709 13:05:03.109841 21112 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.359s	user 0.000s	sys 0.000s
I20260709 13:05:03.110203 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1276) took 1104 ms. Trace:
I20260709 13:05:03.111266 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:03.112196 15088 raft_consensus.cc:2376] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14. Re-sending same reply.
I20260709 13:05:03.110718 15232 rpcz_store.cc:276] 0709 13:05:02.005669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.008943 (+  3274us) service_pool.cc:224] Handling call
0709 13:05:02.340527 (+331584us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:02.340712 (+   185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.341287 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:02.341332 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.341908 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:05:03.059741 (+717833us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:03.059800 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.059904 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.109822 (+ 49918us) raft_consensus.cc:1685] finished
0709 13:05:03.109850 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.110129 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3345,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"spinlock_wait_cycles":2835712,"thread_start_us":717327,"threads_started":1,"wal-append.queue_time_us":765421}
I20260709 13:05:03.113059 21027 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:03.114207 20604 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.114688 20604 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.115063 20604 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.836124 21114 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.236s
I20260709 13:05:03.119196 20982 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:02.921819 20870 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:03.048429 21018 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:03.121878 20870 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.122566 21018 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.122828 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:03.123790 20834 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.005s	sys 0.027s
W20260709 13:05:03.124979 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 972056us
I20260709 13:05:03.126235 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.127714 20875 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:02.876736 21108 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:02.936779 21008 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:03.053903 20894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.000s	sys 0.025s
I20260709 13:05:03.143229 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1274) took 1138 ms. Trace:
I20260709 13:05:03.104928 15080 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:03.076454 20981 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.132879 20834 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:03.142894 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1283) took 1134 ms. Trace:
I20260709 13:05:03.158571 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.165769 15842 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:03.130472 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.135324 21038 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.143853 15228 rpcz_store.cc:276] 0709 13:05:02.004974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.031455 (+ 26481us) service_pool.cc:224] Handling call
0709 13:05:02.062619 (+ 31164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.083716 (+ 21097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.083780 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.103978 (+ 20198us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.757161 (+653183us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:02.757209 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.757271 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.142956 (+385685us) raft_consensus.cc:1685] finished
0709 13:05:03.142984 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.143157 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1184,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":14474,"thread_start_us":589884,"threads_started":1,"wal-append.queue_time_us":978783}
I20260709 13:05:03.146920 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.169631 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.175930 20604 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.124711 21018 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.128687 20875 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:05:03.128276 20975 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:03.202732 20975 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:03.203224 20975 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:03.154858 20976 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:05:03.179301 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:03.179514 20957 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.180397 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1273) took 1175 ms. Trace:
I20260709 13:05:03.181705 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1397) took 1539 ms. Trace:
I20260709 13:05:03.184967 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.207317 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.151122 15236 rpcz_store.cc:276] 0709 13:05:02.008008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.156654 (+148646us) service_pool.cc:224] Handling call
0709 13:05:02.222096 (+ 65442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.346667 (+124571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.346710 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.370876 (+ 24166us) log.cc:844] Serialized 26 byte log entry
0709 13:05:03.063092 (+692216us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:03.063151 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.063264 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.142635 (+ 79371us) raft_consensus.cc:1685] finished
0709 13:05:03.142663 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.142823 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":476,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":27782,"thread_start_us":691731,"threads_started":1,"wal-append.queue_time_us":762601}
I20260709 13:05:03.128659 21108 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:03.129248 21070 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.224040 21070 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.224674 21070 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:03.227219 21070 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:03.227988 21070 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.228348 21070 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.229287 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.230129 15086 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
W20260709 13:05:03.231081 15054 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.018s
W20260709 13:05:03.231426 15054 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.011s	sys 0.018s
W20260709 13:05:03.231688 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 526069us
I20260709 13:05:03.211056 15083 rpcz_store.cc:276] 0709 13:05:01.642706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.931731 (+289025us) service_pool.cc:224] Handling call
0709 13:05:01.932097 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.960507 (+ 28410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.960586 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.023240 (+ 62654us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.835751 (+812511us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:02.835810 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.835907 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.181434 (+345527us) raft_consensus.cc:1685] finished
0709 13:05:03.181463 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.181633 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4553,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":753343,"threads_started":1,"wal-append.queue_time_us":1111727}
I20260709 13:05:03.198562 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.201788 21081 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:03.206425 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:03.212632 15079 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:03.127974 20835 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.000s
I20260709 13:05:03.244688 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1403) took 1568 ms. Trace:
I20260709 13:05:03.101379 21080 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.210970 15234 rpcz_store.cc:276] 0709 13:05:02.004546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.031569 (+ 27023us) service_pool.cc:224] Handling call
0709 13:05:02.066024 (+ 34455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.091397 (+ 25373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.091451 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.117768 (+ 26317us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.377834 (+260066us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b48002610d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:05:02.731899 (+354065us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:02.731961 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.732059 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.180001 (+447942us) raft_consensus.cc:1685] finished
0709 13:05:03.180022 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.180322 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26806,"raft.run_cpu_time_us":2745,"raft.run_wall_time_us":282801,"spinlock_wait_cycles":19825536,"thread_start_us":548691,"threads_started":1,"wal-append.queue_time_us":985806}
I20260709 13:05:03.228669 21070 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:03.257774 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51460.1us
I20260709 13:05:03.189489 20894 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:03.266865 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55070.1us
I20260709 13:05:03.204167 20975 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.159705 20995 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:05:03.143628 21104 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:05:03.278525 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:03.277184 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.277725 15843 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:03.324131 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:03.325066 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.325376 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:03.326328 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.327397 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.328383 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.245297 15084 rpcz_store.cc:276] 0709 13:05:01.676322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.072152 (+395830us) service_pool.cc:224] Handling call
0709 13:05:02.072537 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.081740 (+  9203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.081770 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.082359 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.844491 (+762132us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:02.844552 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.844727 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.244421 (+399694us) raft_consensus.cc:1685] finished
0709 13:05:03.244449 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.244617 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8883,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":726013,"threads_started":1,"wal-append.queue_time_us":1018382}
I20260709 13:05:03.329116 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:03.282709 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.331863 15079 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:03.253559 20998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.012s	sys 0.016s
I20260709 13:05:03.289407 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.323764 15083 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:03.183980 20981 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.341990 15081 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
W20260709 13:05:03.282779 21060 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.000s
I20260709 13:05:03.197415 21038 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.333834 20916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.004s	sys 0.038s
I20260709 13:05:03.360100 15080 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:03.348418 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.361094 15842 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:03.359926 21100 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.000s	user 0.009s	sys 0.005s
W20260709 13:05:03.349138 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.361850 21100 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.362689 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.363693 21038 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.363880 21100 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:03.354740 21060 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.355000 20975 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:03.366307 21060 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.260605 21018 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:03.370461 21018 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.360562 20916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.005s	sys 0.038s
I20260709 13:05:03.352245 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.373947 20916 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:03.278154 20957 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.270154 21098 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:03.344103 21053 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.279374 21124 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:03.259349 20835 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.341466 20998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.013s	sys 0.017s
I20260709 13:05:03.284610 21080 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:03.279599 21042 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:03.342219 15086 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:03.380483 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.021s	sys 0.027s
I20260709 13:05:03.380611 15843 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:03.383816 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.022s	sys 0.027s
I20260709 13:05:03.283051 21139 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 13:05:03.349578 20934 consensus_queue.cc:1243] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:05:03.386057 15235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.012s
W20260709 13:05:03.391109 15235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.004s	sys 0.012s
W20260709 13:05:03.386931 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 684604us
I20260709 13:05:03.344892 21008 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.391404 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.393539 15087 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:03.395522 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1401) took 1719 ms. Trace:
I20260709 13:05:03.381831 21098 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.411394 21098 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.412916 21127 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:05:03.411222 15085 rpcz_store.cc:276] 0709 13:05:01.675725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.054480 (+378755us) service_pool.cc:224] Handling call
0709 13:05:02.064705 (+ 10225us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.084457 (+ 19752us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:02.337596 (+253139us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18003ff9a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:05:02.337951 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.851015 (+513064us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:02.873038 (+ 22023us) log.cc:844] Serialized 52 byte log entry
0709 13:05:02.873250 (+   212us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:03.173992 (+300742us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b18003ff9a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:05:03.392306 (+218314us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.395238 (+  2932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.395311 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.395347 (+    36us) raft_consensus.cc:1685] finished
0709 13:05:03.395359 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.395472 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32819,"prepare.queue_time_us":258109,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":9291,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":33446,"spinlock_wait_cycles":48762496,"thread_start_us":730629,"threads_started":2,"wal-append.queue_time_us":794450}
W20260709 13:05:03.401479 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52726.2us
I20260709 13:05:03.417881 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.402209 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.421195 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.422243 20834 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:03.423976 20834 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:03.424360 20834 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.402240 21080 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:03.426138 21140 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:03.426129 21080 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.428390 21080 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:03.431630 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.381937 21124 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:03.405035 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.408736 21018 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.385480 21114 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.426688 21140 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.400269 20791 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:03.433154 15843 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:03.399528 20934 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:03.440102 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.015s	sys 0.027s
W20260709 13:05:03.473519 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 350834us
I20260709 13:05:03.473572 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.396414 21042 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:03.384624 21053 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.477730 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 541778us
I20260709 13:05:03.424697 20834 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.478689 15086 raft_consensus.cc:2410] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:03.478998 15845 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:03.478749 15842 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:03.404115 21100 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.480955 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:03.480937 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.484174 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.487599 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:03.482921 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:03.488418 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:03.489204 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:03.492830 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:03.493475 15161 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
W20260709 13:05:03.480974 21129 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:03.497392 15847 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:05:03.497695 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1353) took 1429 ms. Trace:
W20260709 13:05:03.499714 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 624041us
I20260709 13:05:03.477533 20934 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:03.500075 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59017.9us
I20260709 13:05:03.505340 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.501157 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:03.519035 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
W20260709 13:05:03.511584 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.477100 21140 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.537117 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.538616 20976 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:03.539140 20976 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:03.539579 20976 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:03.491163 21060 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:05:03.491108 21038 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:05:03.482482 21018 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.569218 15128 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.569129 21018 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.570513 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:03.570921 21111 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:03.571446 21018 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.571522 21111 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:03.550616 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.572521 15085 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:03.573091 21108 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:03.574365 20981 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.574946 20981 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.575229 21108 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:03.575541 20981 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.575881 21108 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:03.576392 21108 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:03.575918 20934 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:03.577000 21153 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:03.577472 21153 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.577801 21153 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:03.577826 20934 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.578665 21098 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.579790 21115 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.498128 15157 rpcz_store.cc:276] 0709 13:05:02.067969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.184098 (+116129us) service_pool.cc:224] Handling call
0709 13:05:02.184370 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.184591 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:02.725842 (+541251us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.497342 (+771500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.497393 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:03.497443 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.497520 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.497644 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2362,"prepare.queue_time_us":629484,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":6152,"raft.queue_time_us":179877,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":1261548,"threads_started":3,"wal-append.queue_time_us":380167}
I20260709 13:05:03.490239 20935 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.555279 15052 leader_election.cc:400] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:05:03.581521 15052 leader_election.cc:403] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:05:03.555819 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54744.9us
I20260709 13:05:03.558511 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.582933 15086 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
W20260709 13:05:03.574985 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 94190us
I20260709 13:05:03.558823 21109 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.559101 20791 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.560055 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:03.562042 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.492599 20604 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.593710 21008 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:03.593993 21154 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:03.594434 21154 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:03.594898 21154 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:03.500236 21144 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.596253 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2239) took 1969 ms. Trace:
I20260709 13:05:03.536578 21118 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.102s
I20260709 13:05:03.596789 21124 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.486142 20975 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.586156 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80581.1us
I20260709 13:05:03.482718 21114 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.597522 21118 consensus_queue.cc:1243] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 17.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:03.576781 21108 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:03.603361 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.608873 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:03.596899 15235 rpcz_store.cc:276] 0709 13:05:01.626355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.645350 (+ 18995us) service_pool.cc:224] Handling call
0709 13:05:01.693150 (+ 47800us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.745493 (+1052343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.745525 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:02.754523 (+  8998us) log.cc:844] Serialized 52 byte log entry
0709 13:05:03.396739 (+642216us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:03.396793 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.396926 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.595980 (+199054us) raft_consensus.cc:1685] finished
0709 13:05:03.596012 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.596183 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39687,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":160,"thread_start_us":1325122,"threads_started":2,"wal-append.queue_time_us":688250}
I20260709 13:05:03.598481 21018 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:03.623409 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63455us
I20260709 13:05:03.631451 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.610289 21098 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:03.598167 20935 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.650396 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:03.651032 15233 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:03.651201 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:03.652544 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:03.633685 21124 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:03.596333 21144 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.638103 21139 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:03.607754 20975 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.652561 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49313.7us
I20260709 13:05:03.661278 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:03.602744 21109 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.646358 20835 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.622910 20976 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.713980 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:03.616106 20791 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.717434 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 69492.5us
I20260709 13:05:03.736689 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:03.738479 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 146342us
W20260709 13:05:03.739140 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 147502us
I20260709 13:05:03.629884 21042 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:03.739732 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 76472.6us
I20260709 13:05:03.718166 21044 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:03.717432 21151 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:03.717953 21141 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.718245 20981 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.740309 21027 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.759559 21044 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:03.772238 21100 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:03.769385 21148 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:03.768189 21150 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:03.779656 21151 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.786029 21144 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.786641 21151 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.775071 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:03.794337 20791 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.794538 20976 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:03.795605 14985 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 369) took 22731 ms (client timeout 15000 ms). Trace:
I20260709 13:05:03.797957 21109 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:03.796022 14985 rpcz_store.cc:269] 0709 13:04:41.064323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.064737 (+   414us) service_pool.cc:224] Handling call
0709 13:05:03.795364 (+22730627us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:52.591949 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:52.684157 (+ 92208us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:52.684240 (+    83us) write_op.cc:435] Acquired schema lock
0709 13:04:52.684275 (+    35us) tablet.cc:662] Decoding operations
0709 13:04:52.799440 (+115165us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:52.799553 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:52.799583 (+    30us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:04:52.832157 (+ 32574us) tablet.cc:701] Row locks acquired
0709 13:04:52.832224 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:04:52.832447 (+   223us) write_op.cc:270] Start()
0709 13:04:52.832598 (+   151us) write_op.cc:276] Timestamp: P: 1783602292832373 usec, L: 0
0709 13:04:52.832649 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:04:52.833511 (+   862us) log.cc:844] Serialized 9855 byte log entry
0709 13:04:55.777488 (+2943977us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:04:55.896025 (+118537us) op_driver.cc:464] REPLICATION: finished
0709 13:04:56.036961 (+140936us) write_op.cc:301] APPLY: starting
0709 13:04:56.042502 (+  5541us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:57.529714 (+1487212us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:57.529760 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:57.808441 (+278681us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:57.811679 (+  3238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:04:57.841550 (+ 29871us) write_op.cc:312] APPLY: finished
0709 13:04:58.004079 (+162529us) log.cc:844] Serialized 220 byte log entry
0709 13:04:58.140916 (+136837us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:58.141134 (+   218us) write_op.cc:454] Released schema lock
0709 13:04:58.141214 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:58.145419 (+  4205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:58.156272 (+ 10853us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7018,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":117863,"apply.run_cpu_time_us":59686,"apply.run_wall_time_us":2125923,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108777,"num_ops":25,"prepare.queue_time_us":275567,"prepare.run_cpu_time_us":21498,"prepare.run_wall_time_us":443609,"raft.queue_time_us":112869,"raft.run_cpu_time_us":56999,"raft.run_wall_time_us":92153,"replication_time_us":3063203,"spinlock_wait_cycles":5048576,"thread_start_us":603752,"threads_started":5,"wal-append.queue_time_us":442411,"wal-append.run_cpu_time_us":39440,"wal-append.run_wall_time_us":4825546}]]}
I20260709 13:05:03.800244 21053 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.656446 21111 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.761005 21139 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:03.801812 20957 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:03.801942 21168 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:03.765647 21115 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:05:03.774690 21157 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:05:03.835116 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 98828.2us
I20260709 13:05:03.763856 21038 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:03.782281 21100 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.867929 21100 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:03.781898 21114 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:03.872896 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 211679us
I20260709 13:05:03.871271 21038 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:03.831701 21127 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.887195 15842 raft_consensus.cc:1217] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Deduplicated request from leader. Original: 14.14->[16.15-16.15]   Dedup: 16.15->[]
I20260709 13:05:03.870644 21144 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:03.887918 15842 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:05:03.890931 21147 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:03.892443 21171 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:03.864881 14979 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 6 to 10, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:03.893965 21140 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.907886 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:03.831777 20981 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:05:03.897382 21038 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.883951 21111 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.783102 21044 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:03.912699 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:03.863559 21151 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:05:03.917622 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1412) took 1798 ms. Trace:
I20260709 13:05:03.923626 15087 rpcz_store.cc:276] 0709 13:05:02.119099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.203452 (+ 84353us) service_pool.cc:224] Handling call
0709 13:05:03.917348 (+1713896us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54001b17d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:03.917542 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734128384}
I20260709 13:05:03.927095 15157 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:03.904870 21144 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:03.928100 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:03.933560 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 172596us
W20260709 13:05:03.936698 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 180030us
I20260709 13:05:03.935513 21044 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.941938 21141 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:03.943555 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:03.944224 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73738.8us
I20260709 13:05:03.931643 21038 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.973150 21144 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.974586 21144 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.931788 21147 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.931788 21150 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:03.977504 21150 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:03.978094 21150 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:03.983232 21127 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:03.983794 21127 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.985263 21082 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:03.985678 21082 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:03.986063 15162 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:03.986562 21082 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.987844 21082 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.987673 21108 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:03.988866 15163 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:05:03.989955 21140 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:03.990347 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:03.990634 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.991394 15155 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:05:03.953984 21158 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:03.955850 21154 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.006366 21154 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:03.956683 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.007603 15154 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 12: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:04.008183 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71126.6us
I20260709 13:05:04.009449 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:03.959862 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:03.948028 21070 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.002s
I20260709 13:05:04.012936 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:04.015858 21165 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.241s	user 0.000s	sys 0.004s
I20260709 13:05:04.016095 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.016724 15087 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:04.016965 21177 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:04.021350 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.021656 15233 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:03.949899 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:04.027379 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.027978 15084 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.028296 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:03.998323 21064 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.028992 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:04.028937 15080 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:04.034607 15155 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:04.034953 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.036408 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.037029 15087 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.038561 21184 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.038887 20791 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:04.039053 21184 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.039392 20791 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.039543 21184 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.039830 20791 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:04.041649 15080 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:04.047809 21027 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:04.048394 21027 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.054780 21153 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:04.055932 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45714us
I20260709 13:05:04.058049 21022 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:04.058488 21022 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:04.059326 21022 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.060402 21022 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:04.063477 21100 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.065963 21118 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:04.066413 21118 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:04.067373 21118 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.021200 21147 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:05:04.069182 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 109429us
W20260709 13:05:04.069617 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.071098 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.071781 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:04.072036 21176 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:04.072561 21180 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:04.073892 15081 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 13:05:04.074021 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 124207us
W20260709 13:05:04.077718 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47577.8us
I20260709 13:05:04.081652 21141 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.068671 21118 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.084460 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.026993 20985 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.080626 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.086347 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.093767 15234 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:04.037608 21124 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:05:04.101858 21141 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:04.021803 21070 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:04.121371 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.124532 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 95600.9us
I20260709 13:05:04.125952 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.135322 21166 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:04.119971 20985 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.139987 21027 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.140676 20985 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.141192 20985 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.145466 15160 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:04.146629 21070 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.147392 21027 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.048107 14979 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 10 to 8, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 8 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.149693 21108 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.149801 21172 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:04.150000 20834 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.003s
I20260709 13:05:04.150431 21172 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.074930 21064 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.151554 21100 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:04.150923 21172 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.153469 21064 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.150892 20834 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.158764 15844 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:04.161592 21172 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.151580 14979 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 6 to 7, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 7 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:04.135166 21183 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:05:04.169684 21183 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:04.170045 21183 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:04.170881 21183 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.172088 21183 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:04.171285 21172 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.166779 20975 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.198777 21150 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.201118 21150 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:04.168102 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78933.7us
I20260709 13:05:04.166466 20834 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:04.208683 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.183753 21153 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.210582 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.211285 15082 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:05:04.213781 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 87869.1us
I20260709 13:05:04.179937 15087 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
W20260709 13:05:04.185685 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 111477us
I20260709 13:05:04.217756 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:04.184026 20791 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.192989 20935 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.185482 20985 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:04.185351 21170 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.412s	user 0.000s	sys 0.003s
I20260709 13:05:04.167418 21174 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:05:04.204079 21038 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.206422 15088 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:05:04.219163 15084 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:04.235090 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:04.235381 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 149137us
I20260709 13:05:04.246227 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:04.251812 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.252751 21168 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:04.247406 15810 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:04.253450 21150 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:04.253633 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.254887 21150 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.250497 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.255076 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.255785 15080 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:04.256292 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 177247us
I20260709 13:05:04.256418 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.257059 20604 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.257953 20604 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.258456 20604 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.248078 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:04.258893 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:04.275249 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.259119 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.289996 15160 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:04.293018 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.293807 15084 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:04.295992 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.296885 15083 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:04.233922 21018 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.005s
W20260709 13:05:04.297806 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51686us
I20260709 13:05:04.233553 20975 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.303776 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.304728 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:04.304993 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.305317 15844 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.305600 15088 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.306981 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.308985 21080 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.309132 21151 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.299803 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.300832 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.249192 20995 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:04.302842 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:04.303006 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44226.2us
I20260709 13:05:04.303599 20985 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.219425 21118 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.310921 15154 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.258886 20604 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:04.324927 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.326885 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.323350 15160 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:04.314848 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.309551 21080 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.297914 21038 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.381219 15127 leader_election.cc:400] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
I20260709 13:05:04.309544 21151 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.311285 21018 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
W20260709 13:05:04.377576 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.310248 21078 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:04.330889 15843 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:04.341001 15088 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:04.398878 15127 leader_election.cc:403] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:04.328661 15087 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:04.333899 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85901.3us
I20260709 13:05:04.297310 21042 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:04.368440 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56297.8us
W20260709 13:05:04.388927 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 308377us
W20260709 13:05:04.408896 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 75727.7us
W20260709 13:05:04.409142 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84481us
W20260709 13:05:04.410140 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 95388us
W20260709 13:05:04.403072 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.074s (SENT) [suppressed 38 similar messages]
W20260709 13:05:04.418205 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.422360 15087 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:04.426888 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.346885 21153 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.428556 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.392352 21150 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.449244 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.400570 20995 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:04.405229 21151 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.451507 15839 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:04.454049 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.448314 15088 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:04.449503 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.453404 20981 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:04.455869 20981 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:04.457134 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.457008 20981 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.458278 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.459225 20983 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:04.459380 20791 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.459621 20983 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.460042 20983 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:04.449235 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:04.453416 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:04.464591 15845 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.413491 20998 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:04.462503 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.346143 21070 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:05:04.471798 15844 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:04.471951 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.476821 15086 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
W20260709 13:05:04.477365 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.436048 21153 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:04.484079 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.377664 21127 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.345484 21081 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:05:04.460072 20791 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.330284 21118 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.513434 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.513545 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.514182 15236 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:04.514264 21082 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:04.565999 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52531.9us
I20260709 13:05:04.566901 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:04.514588 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 146155us
I20260709 13:05:04.515702 21109 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.516909 15847 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:04.529564 21080 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:04.533250 20995 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.529657 15234 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:04.558486 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45150.7us
I20260709 13:05:04.574079 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:04.487352 21042 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.486047 20834 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.491698 21153 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.580520 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.583319 15845 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
W20260709 13:05:04.566972 21111 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 13:05:04.496307 21174 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.592815 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:04.514143 20998 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.580443 20870 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:04.600950 15156 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:04.601080 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.601281 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.629290 15159 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:04.602092 15813 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:04.539307 21194 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.614682 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:04.632917 15086 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:04.635917 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:04.587841 20982 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.615952 15845 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:04.599812 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.583434 21168 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:04.625885 21127 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:04.626063 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:04.588189 21195 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.643507 15163 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:04.626590 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:04.621662 21184 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.582s	user 0.004s	sys 0.007s
I20260709 13:05:04.658473 15162 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:04.647362 15839 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:04.593526 21042 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.579562 21082 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.645705 20870 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:04.645202 21168 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.645171 20982 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:04.679600 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.680559 15160 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:04.656948 20894 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:04.650336 15234 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:05:04.645241 21194 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.662027 15159 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:04.683013 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.683748 15159 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:04.623893 20875 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:04.668200 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.603178 20834 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.714877 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.712881 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:04.647048 21144 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13.
I20260709 13:05:04.647339 21080 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.717382 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:04.657814 21195 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.629374 21174 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.725327 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.724258 15159 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:04.733819 15839 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:04.686426 21168 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:04.745401 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.742496 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.685874 20982 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.752444 15154 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:04.754711 15234 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:04.669950 21184 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:04.740242 21195 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.762434 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:04.773480 15080 consensus_meta.cc:287] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.006s
I20260709 13:05:04.774745 15231 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:04.751124 21175 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.751327 20894 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:04.754179 20875 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:04.778586 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.788115 15154 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:04.779222 15080 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:04.781243 15233 raft_consensus.cc:2368] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:05:04.776507 15128 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:04.783735 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:04.793161 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:04.793919 15230 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:04.801486 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:04.804973 21192 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:04.814023 21082 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.815680 20983 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.817936 20983 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:04.818092 21190 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.365s	user 0.003s	sys 0.000s
I20260709 13:05:04.818570 20983 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:04.819020 20983 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:04.819895 20983 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.821823 21174 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.828477 15161 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:05:04.858844 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.832993 21181 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:04.833266 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.795032 21175 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.833847 20875 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.795328 21150 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.865056 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.880573 15082 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:04.835191 20998 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.835592 20834 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.835709 21118 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.836984 21194 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.894626 15161 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:04.841985 21070 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:04.842577 20985 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:05:04.897270 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.845964 21193 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.600s	user 0.003s	sys 0.000s
I20260709 13:05:04.847244 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.849500 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.853161 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:04.854282 21197 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:05:04.766717 21038 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:04.858376 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.905639 15230 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:04.860205 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.906929 15081 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:04.907840 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.861455 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.909776 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.911165 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.912904 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:04.908473 15081 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:04.917682 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.920508 15087 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:05:04.921021 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.867462 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:04.921926 15232 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:04.922909 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.924330 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.787180 20870 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.871573 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.928524 15084 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:04.929953 15084 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:04.931344 15084 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:04.874725 15163 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:04.935352 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:04.936101 15163 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:04.882670 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:04.937263 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54686.3us
I20260709 13:05:04.938769 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:04.883760 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.887876 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:04.893064 15154 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:05:04.940510 21194 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:04.941296 21194 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:04.942443 21194 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.946580 20834 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:04.948467 20834 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.855150 21109 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:04.894290 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:04.953327 21109 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:04.953889 21109 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:04.899752 15234 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:04.903774 15085 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:04.958583 20834 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.959214 20834 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:04.864710 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.940594 15154 pending_rounds.cc:85] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 5
I20260709 13:05:04.969404 15154 pending_rounds.cc:107] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:05:04.969782 15154 raft_consensus.cc:2889] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.844435 21153 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.840497 21151 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.972718 21153 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.976114 21151 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:04.976951 15159 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:04.952469 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:04.980192 21104 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.981192 21104 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:04.983886 21104 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:04.984145 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.972563 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:04.989022 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:04.990607 21120 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:04.985122 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.991160 21120 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.991740 21120 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:04.991930 21104 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.992580 21104 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:04.993085 21104 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.881302 21174 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:05.004271 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.011405 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:05.012267 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79448.4us
I20260709 13:05:05.018970 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.022320 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.026319 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:04.805584 20894 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.027567 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.028573 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:05.029196 15082 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:05.030197 15081 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:04.905836 21070 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:05.031648 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.053579 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:05.060667 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:05.030923 21191 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:05:05.062489 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.068171 21042 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.068681 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.069532 15229 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
W20260709 13:05:05.070619 21203 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:05:05.071391 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.071985 15154 raft_consensus.cc:2376] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14. Re-sending same reply.
I20260709 13:05:05.072757 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.065315 20995 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:05.072677 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.076382 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.077574 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.078614 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.070555 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.082080 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.082865 15229 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:05.083289 20604 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.724s	user 0.006s	sys 0.002s
I20260709 13:05:05.083642 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.085234 15156 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.087790 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.095427 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:05.058552 20791 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.000s
W20260709 13:05:05.111867 21196 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.848s	user 0.005s	sys 0.000s
I20260709 13:05:05.115391 15082 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:05.116643 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:05.116679 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.096889 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1426) took 1081 ms. Trace:
I20260709 13:05:05.020561 21154 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:05:05.120126 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:05.065249 21183 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.001s
I20260709 13:05:05.116115 20791 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:05.131280 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.058923 20834 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.162973 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.163851 15154 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:05:05.164311 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.164813 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.123669 20976 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:05.160687 15230 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:05.118700 15079 rpcz_store.cc:276] 0709 13:05:04.015231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.015349 (+   118us) service_pool.cc:224] Handling call
0709 13:05:04.016307 (+   958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.016687 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.016715 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.017166 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:05:04.163476 (+146310us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b4800080358. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:05:04.245529 (+ 82053us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:04.245570 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.245657 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:04.845303 (+599646us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:05.096593 (+251290us) raft_consensus.cc:1685] finished
0709 13:05:05.096624 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.096793 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17284,"raft.queue_time_us":1220,"raft.run_cpu_time_us":929,"raft.run_wall_time_us":141976,"spinlock_wait_cycles":121058432,"thread_start_us":227978,"threads_started":1,"wal-append.queue_time_us":228421}
W20260709 13:05:05.168028 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.170055 21120 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.172458 21120 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:05.160463 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:05.173835 21200 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:05:05.176429 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.177203 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.177363 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:05.177894 21214 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:05:05.177990 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.178594 21175 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:05.179096 15845 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:05.179677 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.180367 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.181064 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.180991 21070 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.120978 21150 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:05.184680 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.185707 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.095983 20604 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.187067 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.126282 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.117238 21109 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.153097 20982 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.390050 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.390458 15229 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:05.159399 21140 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:05.218712 15841 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.392244 15158 raft_consensus.cc:1217] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.14->[16.15-16.15]   Dedup: 16.15->[]
W20260709 13:05:05.373874 15162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.031s
W20260709 13:05:05.392975 15162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.032s
W20260709 13:05:05.373505 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.007s	sys 0.027s
W20260709 13:05:05.393920 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.008s	sys 0.028s
W20260709 13:05:05.272740 15051 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.688s (SENT)
W20260709 13:05:05.394361 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 591021us
W20260709 13:05:05.394857 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.394723 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 122049us
I20260709 13:05:05.391247 15159 raft_consensus.cc:2376] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21. Re-sending same reply.
I20260709 13:05:05.391708 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1377) took 1134 ms. Trace:
I20260709 13:05:05.398778 15157 rpcz_store.cc:276] 0709 13:05:04.257417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.257569 (+   152us) service_pool.cc:224] Handling call
0709 13:05:04.257887 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.309040 (+ 51153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.309072 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.309591 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:05:04.645570 (+335979us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:04.645612 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.645703 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.391498 (+745795us) raft_consensus.cc:1685] finished
0709 13:05:05.391517 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.391657 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40552,"raft.queue_time_us":52037,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":11139,"thread_start_us":335618,"threads_started":1,"wal-append.queue_time_us":445742}
W20260709 13:05:05.177258 21204 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.285s	user 0.000s	sys 0.000s
I20260709 13:05:05.391343 15154 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:05.395639 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:05.407325 15158 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:05:05.412820 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.166360 21183 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:05:05.413991 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.409744 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.414305 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.415040 15157 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:05.416611 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.418169 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.419286 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.419554 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.419615 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.421162 15848 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.422045 15844 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 12: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:05.422638 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.422827 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1376) took 1344 ms. Trace:
I20260709 13:05:05.423123 15159 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
W20260709 13:05:05.303584 20916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.032s	sys 0.047s
I20260709 13:05:05.428508 15228 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:05.420454 15154 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:05.430202 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.432200 15154 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:05.427107 15234 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:05.434466 15232 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:05.435781 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.411638 21189 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.438675 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.389719 21184 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:05.442873 15840 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:05.444217 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.445422 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.439443 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.439770 15843 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:05.434052 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.423369 15155 rpcz_store.cc:276] 0709 13:05:04.078883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.079035 (+   152us) service_pool.cc:224] Handling call
0709 13:05:04.079353 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:04.083977 (+  4624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.084010 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:04.103220 (+ 19210us) log.cc:844] Serialized 52 byte log entry
0709 13:05:04.247051 (+143831us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:04.247093 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.247156 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.095015 (+847859us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:05.391911 (+296896us) raft_consensus.cc:1685] finished
0709 13:05:05.391930 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.422754 (+ 30824us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1804,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4274,"spinlock_wait_cycles":104088448,"thread_start_us":69869,"threads_started":1,"wal-append.queue_time_us":117479}
I20260709 13:05:05.447659 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.448411 15155 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:05.449499 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.450443 15155 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:05.451375 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.452049 15155 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:05.440922 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.453318 15844 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:05.408389 21150 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:05.442906 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 492404us
I20260709 13:05:05.425185 21192 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:05.432957 20916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.034s	sys 0.047s
I20260709 13:05:05.446612 15229 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:05.453687 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:05.469362 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.469697 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.470921 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.389596 21104 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.292680 20791 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.158s
I20260709 13:05:05.457360 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.459167 15847 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:05.425510 21140 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:05.460304 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.462008 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.464412 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.437043 21154 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.454231 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:05.454697 21168 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 13:05:05.443195 21174 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:05.457388 20834 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:05.479172 15234 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:05.482240 15155 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:05:05.484760 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.485545 15229 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:05:05.472229 20998 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:05.488499 15237 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.489245 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.477581 21192 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:05.482829 21153 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:05.495438 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.461169 21042 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
W20260709 13:05:05.496949 15235 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.001s
I20260709 13:05:05.499600 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.501817 15844 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:05.502193 15235 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:05.505537 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.508177 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.508600 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.510842 15233 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:05.516230 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:05.517439 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.505625 20916 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:05.521822 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.483125 21154 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.524714 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.530416 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.526093 21153 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:05.484673 21174 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.501359 21224 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:05.492512 21212 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:05:05.490444 21168 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.553418 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.555980 15236 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:05.559866 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.524622 21151 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:05.561455 15229 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 12: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:05.566111 15236 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:05.567353 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.572801 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.569651 15231 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:05.505190 21150 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:05.589640 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.592900 15051 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.940s (SENT)
I20260709 13:05:05.593514 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:05.599685 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 100153us
I20260709 13:05:05.600564 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:05.601821 21153 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:05.602206 21153 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.602551 21153 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.602602 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:05.604583 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.605435 21042 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:05.570922 21221 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:05.454100 21189 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.568221 20982 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:05.569079 21109 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.537386 21140 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.620060 15844 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.607270 20976 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:05.643653 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 181704us
I20260709 13:05:05.566524 21151 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.581229 21192 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.657724 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.648969 21104 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:05.673262 15814 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:05.675742 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2533) took 1076 ms. Trace:
I20260709 13:05:05.676149 15162 rpcz_store.cc:276] 0709 13:05:04.599169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.746365 (+147196us) service_pool.cc:224] Handling call
0709 13:05:04.758536 (+ 12171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.765416 (+  6880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.765461 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.778156 (+ 12695us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.393515 (+615359us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:05.393569 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.393675 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.675523 (+281848us) raft_consensus.cc:1685] finished
0709 13:05:05.675547 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.675686 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6459,"raft.queue_time_us":11212,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4682,"thread_start_us":580317,"threads_started":1,"wal-append.queue_time_us":688152}
I20260709 13:05:05.657729 21042 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.681708 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.647060 21236 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:05:05.680986 21223 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:05:05.679832 20995 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:05.690111 21174 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:05.679579 20834 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:05.689239 21217 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:05:05.693030 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:05.685595 21222 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.165s	user 0.001s	sys 0.001s
I20260709 13:05:05.697914 21151 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.692207 20976 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.683029 21241 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.659377 21229 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:05:05.705699 21229 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:05.706113 21229 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:05.707000 21229 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.695585 20834 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:05.727633 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.731041 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.732215 15162 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:05.705052 21168 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.710214 21241 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.694303 21242 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:05:05.737519 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.737818 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.738164 15847 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:05.739403 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.739535 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:05.740146 21195 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.960s	user 0.001s	sys 0.009s
I20260709 13:05:05.740391 21242 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:05.740887 21242 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.741000 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.741338 21242 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.741844 15235 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:05.742525 21189 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:05.743141 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.744671 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.721221 21240 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:05.745272 15235 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:05.746722 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.747507 15235 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:05.748577 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:05.737690 21228 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:05:05.708518 21235 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:05.750427 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:05.750851 15848 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.15-16.15]
W20260709 13:05:05.749176 21226 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:05:05.754211 21241 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.755856 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:05.755836 21235 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:05.726361 20834 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:05.708809 21154 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:05.756438 15229 raft_consensus.cc:2376] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15. Re-sending same reply.
I20260709 13:05:05.765679 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.766438 15229 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:05.767393 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.767999 15229 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:05:05.764966 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:05.758626 21233 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:05:05.749168 15235 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:05.762614 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:05.755892 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68054.3us
I20260709 13:05:05.768978 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.770730 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.770948 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.772519 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.774365 15229 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:05.777400 15235 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:05.740603 21195 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.780965 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2528) took 1183 ms. Trace:
I20260709 13:05:05.781355 15158 rpcz_store.cc:276] 0709 13:05:04.597603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.601956 (+  4353us) service_pool.cc:224] Handling call
0709 13:05:05.392106 (+790150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.416102 (+ 23996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.780762 (+364660us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540033fc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:05.780913 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764593920}
I20260709 13:05:05.782486 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.784142 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2240) took 1365 ms. Trace:
I20260709 13:05:05.784523 15228 rpcz_store.cc:276] 0709 13:05:04.418998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.419208 (+   210us) service_pool.cc:224] Handling call
0709 13:05:04.452775 (+ 33567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.429105 (+976330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.783897 (+354792us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b5400318654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:05.784083 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2243421056}
I20260709 13:05:05.785878 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.786983 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:05.786772 21153 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.787746 15228 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:05.788208 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.788816 15161 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:05.789196 15228 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:05.789525 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.789690 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.790464 15161 raft_consensus.cc:2410] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:05:05.794222 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.794965 15158 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.779645 15126 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.796460 15235 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:05.788976 21153 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.797677 15229 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:05.802830 15229 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:05:05.803527 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.804381 15235 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:05.806380 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.806974 15845 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.812084 20604 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:05.776822 21229 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.796746 21151 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.828492 15229 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:05:05.819036 15845 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:05.830382 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:05.836345 15234 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:05.869707 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 66255.3us
I20260709 13:05:05.871140 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:05.769436 21241 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 13:05:05.875258 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 44994.1us
I20260709 13:05:05.875707 15159 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:05.836160 20604 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.787699 20834 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.883234 15235 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:05.763435 21235 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.892356 15161 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:05.879998 21140 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.881274 21153 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:05.887202 20604 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:05.897285 21153 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:05.898267 21153 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.899170 20604 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:05.900975 21245 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:05.900482 21249 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.909775 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:05.911643 15163 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:05.916468 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:05.917593 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1301) took 1131 ms. Trace:
I20260709 13:05:05.918128 21192 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:05.887431 21042 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Committing config change with OpId 10.8: config changed from index 5 to 8, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:05.917973 15231 rpcz_store.cc:276] 0709 13:05:04.786616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.786905 (+   289us) service_pool.cc:224] Handling call
0709 13:05:04.801790 (+ 14885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.581562 (+779772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.917375 (+335813us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b540043d854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:05.917536 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765383040}
I20260709 13:05:05.924062 15163 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:05:05.925519 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.936661 15158 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:05.945791 21195 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:05.945900 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1440) took 1512 ms. Trace:
I20260709 13:05:05.947260 15086 rpcz_store.cc:276] 0709 13:05:04.433951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.634722 (+200771us) service_pool.cc:224] Handling call
0709 13:05:04.635010 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.635222 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:04.895579 (+260357us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.945469 (+1049890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.945547 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:05.945614 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.945732 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.945841 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1049547,"prepare.queue_time_us":139892,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":1050482,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":229165,"threads_started":1}
I20260709 13:05:05.951931 15847 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:05.952091 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.952672 15844 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:05:05.954463 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:05.893740 21242 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:05.956418 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2348) took 1063 ms. Trace:
I20260709 13:05:05.959913 15084 rpcz_store.cc:276] 0709 13:05:04.892640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.933919 (+ 41279us) service_pool.cc:224] Handling call
0709 13:05:04.934265 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.934503 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:05.052676 (+118173us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.956086 (+903410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.956146 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:05.956194 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.956260 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.956369 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":981606,"prepare.queue_time_us":136972,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":904253}
W20260709 13:05:05.958345 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.958364 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:05.984908 15843 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:06.029503 15844 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.988916 15162 raft_consensus.cc:2368] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16.
I20260709 13:05:05.989176 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:06.030169 15163 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:05.955735 20870 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:05.989931 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.991600 21192 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:05.992587 21244 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:05:06.038213 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.955825 21175 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.997680 21150 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:06.002724 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.003208 21240 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:06.005825 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:06.017886 20998 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.020162 20834 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.022215 15233 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:06.023983 15154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:34534
I20260709 13:05:06.053972 15154 raft_consensus.cc:606] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:06.054337 15154 raft_consensus.cc:614] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:06.025313 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 66067.9us
I20260709 13:05:06.026186 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2529) took 1428 ms. Trace:
I20260709 13:05:06.054936 15156 rpcz_store.cc:276] 0709 13:05:04.597948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.717313 (+119365us) service_pool.cc:224] Handling call
0709 13:05:04.741995 (+ 24682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.095762 (+353767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.025967 (+930205us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540033e854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:06.026129 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788933376}
I20260709 13:05:06.056169 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:06.026455 21216 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.056825 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:06.027462 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68665.4us
I20260709 13:05:06.028142 15847 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:06.028304 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1309) took 1125 ms. Trace:
I20260709 13:05:06.059648 15236 rpcz_store.cc:276] 0709 13:05:04.902620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.563195 (+660575us) service_pool.cc:224] Handling call
0709 13:05:05.565924 (+  2729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.566692 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.028058 (+461366us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b540005b750. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:06.028229 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5490432}
I20260709 13:05:05.988204 21242 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:06.039466 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:06.041695 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.052852 15848 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.057580 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.058449 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:06.070416 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:06.068485 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 197447us
I20260709 13:05:06.074427 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.076211 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.075865 15232 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:06.077399 15228 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:06.052397 20998 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:06.058655 21038 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.051852 21118 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.054092 20935 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.059219 21216 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:06.094808 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.111928 21218 log.cc:927] Time spent T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.625s	user 0.003s	sys 0.000s
I20260709 13:05:06.083506 20834 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:06.113364 20834 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.114379 20834 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.041766 20870 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.117664 20846 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.118243 20846 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.118762 20870 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:06.079690 20975 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:06.120117 20975 consensus_queue.cc:1243] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:06.065497 15845 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:06.068202 20604 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:05:06.121448 21153 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:06.121740 20604 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:06.122244 20604 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:06.120263 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 78654.8us
I20260709 13:05:06.123256 20604 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.125342 20998 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.126628 20998 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.128943 21118 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:06.131615 21118 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.133285 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:06.135308 15848 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:06.136793 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:06.120507 20846 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:06.156154 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 81792.1us
W20260709 13:05:06.157027 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 172151us
I20260709 13:05:06.157177 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.159350 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.124643 20604 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.160079 15087 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:06.047415 21154 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:05:06.161418 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.166024 15086 raft_consensus.cc:2410] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:05:06.162763 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.132382 21235 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.067963 21195 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:05:06.169680 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2343) took 1935 ms. Trace:
I20260709 13:05:06.167860 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2530) took 1569 ms. Trace:
I20260709 13:05:06.167033 15079 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.053799 21175 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.173024 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.171872 15084 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [12.12-14.13]
W20260709 13:05:06.174454 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 581575us
I20260709 13:05:06.171597 15160 rpcz_store.cc:276] 0709 13:05:04.598250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.741744 (+143494us) service_pool.cc:224] Handling call
0709 13:05:04.755420 (+ 13676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.768887 (+ 13467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.768934 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.791360 (+ 22426us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.273610 (+482250us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:05.273650 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.273733 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.025331 (+751598us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b48002001d8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d4490daa1 00007f0d44904b37 00007f0d449049ca 00007f0d449fc26f 00007f0d449fbf37 00007f0d44a1055c 00007f0d44a10508 00007f0d44a10488 00007f0d44a10443 00007f0d44a0f6ec 00007f0d43939a96
0709 13:05:06.111254 (+ 85923us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:06.167573 (+ 56319us) raft_consensus.cc:1685] finished
0709 13:05:06.167614 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.167789 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4988,"raft.run_cpu_time_us":8866,"raft.run_wall_time_us":1255532,"spinlock_wait_cycles":25801728,"thread_start_us":472456,"threads_started":1,"wal-append.queue_time_us":673826}
I20260709 13:05:06.183841 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.184607 15236 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:06.137545 21127 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:06.160920 21240 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:06.191661 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:06.204151 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.209601 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:06.172376 21260 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:05:06.162904 21242 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:06.163982 21111 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:06.214403 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.166884 21144 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:06.237003 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45432.9us
W20260709 13:05:06.233644 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.242523 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.232802 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.234803 15079 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:06.190595 15083 rpcz_store.cc:276] 0709 13:05:04.233850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.466141 (+232291us) service_pool.cc:224] Handling call
0709 13:05:04.492813 (+ 26672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.493060 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:04.895579 (+402519us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.168761 (+1273182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.168819 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:06.169463 (+   644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.169526 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.169629 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":825475,"prepare.queue_time_us":187319,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":817837,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":825185,"threads_started":3,"wal-append.queue_time_us":593825}
I20260709 13:05:06.189180 21195 consensus_queue.cc:1243] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:05:06.237671 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.169258 20846 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:05:06.154443 21038 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.171856 21230 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:05:06.240927 21196 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:06.255121 15087 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:06.250950 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.263244 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.260376 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.261539 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.265200 15079 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:06.233456 21027 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:06.254791 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.236825 21168 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.250515 21151 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.255834 20935 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:06.267558 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.267671 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.217821 21259 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:06.273388 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.265547 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:06.280464 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.15945e+06us
W20260709 13:05:06.281173 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.256076 21229 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.262411 21154 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:06.283491 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 223823us
I20260709 13:05:06.286237 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:06.270933 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 214170us
I20260709 13:05:06.287038 15087 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:06.266232 21144 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:06.291021 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.274569 20975 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:06.309614 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 184241us
I20260709 13:05:06.310652 15083 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.312602 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.317206 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.281054 21249 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:06.320830 15154 pending_rounds.cc:85] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 12
I20260709 13:05:06.328711 15154 pending_rounds.cc:107] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:05:06.329105 15154 raft_consensus.cc:2889] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.272857 21151 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:06.324236 15155 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.318732 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.270728 21267 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.282675 21258 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:06.273301 21168 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:06.335654 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2258) took 1475 ms. Trace:
I20260709 13:05:06.271911 21240 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:05:06.337929 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.282150 21229 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:06.336621 15230 rpcz_store.cc:276] 0709 13:05:04.850705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.078385 (+227680us) service_pool.cc:224] Handling call
0709 13:05:06.325896 (+1247511us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b5400319a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:06.326080 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186830592}
I20260709 13:05:06.338792 15162 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:06.344014 15162 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:05:06.347463 15087 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:06.353982 15084 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:06.355255 21192 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:06.356014 21192 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:06.356793 15156 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:05:06.359062 21192 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:06.360270 15086 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:06.361706 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.364127 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.367033 21194 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:06.367897 21194 consensus_queue.cc:1243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:05:06.368412 21246 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.494s	user 0.007s	sys 0.000s
I20260709 13:05:06.368950 15159 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:06.372059 21127 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:06.372506 21127 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:06.372999 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50074.1us
I20260709 13:05:06.382447 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.377231 20846 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.380806 15157 raft_consensus.cc:2410] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:06.272541 21027 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.328012 20935 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.295462 21261 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.332646 21151 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.321725 20975 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.393864 20846 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.403283 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.405289 15159 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.410851 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:06.411861 15232 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:06.420109 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 129193us
I20260709 13:05:06.424031 15083 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:06.427645 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.338124 21269 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:06.422446 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.432461 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.401010 21082 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:06.442003 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.442797 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:06.452196 15162 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:06.421270 21266 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.459209 15156 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:06.460338 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:06.394307 21127 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.465232 21127 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.473085 21080 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:06.473578 21080 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.474466 21080 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.475158 21229 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:06.475565 21080 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.422863 20985 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:06.476518 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:06.402272 21194 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:06.477767 21194 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.478240 21064 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:06.478816 20846 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.478986 21064 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:06.479579 21064 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:06.444396 21261 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:06.451757 21082 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.463232 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.487666 15080 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.483577 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.447901 21224 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:06.482445 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.490319 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.490688 15232 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:06.491618 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:06.492515 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:06.494313 15203 leader_election.cc:400] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16.
I20260709 13:05:06.494645 15203 leader_election.cc:403] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:06.494786 21229 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.495262 21184 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.495633 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:06.495697 15232 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 5 to 8, 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:06.498823 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:06.499716 21229 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:06.483237 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:06.498297 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70651.1us
I20260709 13:05:06.484709 15159 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
W20260709 13:05:06.487892 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.500711 15162 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:05:06.500762 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53779.3us
I20260709 13:05:06.517755 15080 raft_consensus.cc:2410] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:05:06.504933 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.526207 15842 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:06.528558 15844 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:06.534426 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:06.486618 21264 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:05:06.488314 21261 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.538069 15847 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:06.534539 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:06.560456 15231 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.541867 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.561978 15080 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:06.480932 21038 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:06.542093 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67145.3us
I20260709 13:05:06.487669 21195 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:06.486585 20995 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:06.547151 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1447) took 1575 ms. Trace:
I20260709 13:05:06.578146 15085 rpcz_store.cc:276] 0709 13:05:04.971886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.972973 (+  1087us) service_pool.cc:224] Handling call
0709 13:05:04.973407 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.048080 (+1074673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.048133 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.048718 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.218569 (+169851us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:06.218623 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.218715 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.546889 (+328174us) raft_consensus.cc:1685] finished
0709 13:05:06.546919 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.547087 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1074205,"raft.queue_time_us":1084324,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6197,"thread_start_us":169284,"threads_started":1,"wal-append.queue_time_us":346218}
I20260709 13:05:06.546311 20985 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:06.557502 15848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:06.491497 21082 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:06.562430 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 73044.8us
I20260709 13:05:06.505343 21269 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
W20260709 13:05:06.519625 21263 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:05:06.478274 21194 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:06.589746 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90997.9us
I20260709 13:05:06.485780 21266 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:06.601778 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.567233 21261 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.602646 15085 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:06.603260 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.564399 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.549047 21224 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:06.580821 15848 raft_consensus.cc:606] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:06.581947 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.433745 21144 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.458324 21216 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.480496 21064 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.575520 15841 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:06.576399 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.611630 15080 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:06.612196 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.577970 20995 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.607221 15848 raft_consensus.cc:614] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:06.608521 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.608543 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:06.610473 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70456.7us
W20260709 13:05:06.616331 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52063.2us
I20260709 13:05:06.617837 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:06.621734 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.628744 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.629946 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.543699 20975 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.609148 21269 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:06.630708 15156 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:06.640908 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2253) took 1791 ms. Trace:
I20260709 13:05:06.641309 15237 rpcz_store.cc:276] 0709 13:05:04.849276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.849426 (+   150us) service_pool.cc:224] Handling call
0709 13:05:04.849729 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.488965 (+639236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.640624 (+1151659us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400177354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:06.640821 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468610304}
I20260709 13:05:06.647825 21275 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:05:06.652056 15161 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:06.652503 15161 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:06.639469 21168 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:06.653766 15231 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:06.655576 21281 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.656039 21281 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:06.655894 21151 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.657042 21281 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.657325 20995 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.658396 21281 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.658577 20995 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.659332 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1308) took 1760 ms. Trace:
I20260709 13:05:06.659822 15233 rpcz_store.cc:276] 0709 13:05:04.898834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.555238 (+656404us) service_pool.cc:224] Handling call
0709 13:05:06.022030 (+466792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.053561 (+ 31531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.659072 (+605511us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400318b50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:06.659254 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072712448}
I20260709 13:05:06.613646 21241 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:06.673501 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.608855 21192 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:05:06.668084 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 163278us
I20260709 13:05:06.674973 15231 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:06.667574 15155 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:06.665984 15161 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:06.694866 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.715583 15161 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:06.686187 15233 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:06.716027 15160 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:06.714956 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.719481 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.626482 21175 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.701333 15080 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:06.637498 21111 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:06.726557 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:06.727843 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2271) took 1104 ms. Trace:
W20260709 13:05:06.736269 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.763865 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.766963 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.768148 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.715083 21241 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.770782 21241 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:06.686324 20604 consensus_meta.cc:287] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.004s
I20260709 13:05:06.773710 15083 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:06.777318 15234 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:06.777729 15234 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:06.778735 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:06.779955 21038 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.780185 15081 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:06.780718 21038 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:06.779309 15083 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:06.730679 20985 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:06.632105 21195 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:05:06.664502 21282 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.737352 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.795445 15080 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:06.779362 15084 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:06.793192 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 164529us
I20260709 13:05:06.734539 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:06.799265 15079 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:06.635056 21249 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:06.800385 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.734200 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:06.808699 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 82280.4us
W20260709 13:05:06.810256 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 241586us
I20260709 13:05:06.822284 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.812126 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:06.818096 15085 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:06.800196 15235 rpcz_store.cc:276] 0709 13:05:05.623126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.805338 (+182212us) service_pool.cc:224] Handling call
0709 13:05:05.805682 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.884438 (+ 78756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.727519 (+843081us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b54000ac8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:06.727770 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177728384}
W20260709 13:05:06.811147 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 183746us
I20260709 13:05:06.827623 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.783937 21175 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.833591 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.717041 21269 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:06.836118 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.845768 15156 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:06.723258 21168 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 13:05:06.810576 21038 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:06.848208 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.808101 21249 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:06.835561 21042 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.002s
I20260709 13:05:06.857708 15156 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:06.858160 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.854543 15159 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:06.858876 15162 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:06.855749 15157 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:06.849779 15844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:06.859982 15844 raft_consensus.cc:606] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.860311 15844 raft_consensus.cc:614] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:06.856652 15160 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:06.803043 21275 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:05:06.856092 21174 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
W20260709 13:05:06.854404 21242 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.001s
W20260709 13:05:06.867995 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.853459 21192 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:06.867591 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:06.854249 21038 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:06.877403 21120 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:06.881227 21064 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:06.877923 21120 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.862950 21168 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:06.803987 21224 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.862592 21194 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:06.882843 21120 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.893066 15084 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:06.896327 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.846300 21175 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.869643 21118 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:06.873512 21242 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.885061 21195 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:06.912911 15844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
W20260709 13:05:06.913884 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.858146 21249 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:06.895275 21168 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.889724 21192 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:06.908406 15814 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:06.922015 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 110196us
I20260709 13:05:06.886653 21280 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:06.924544 15844 raft_consensus.cc:606] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.909926 15161 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:05:06.932593 15161 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 9
I20260709 13:05:06.932929 15161 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:05:06.933235 15161 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.910619 21289 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:06.917805 21294 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:06.937160 21278 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:06.937343 21294 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.937613 21278 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:06.937888 21243 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:05:06.937985 21278 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.938330 21294 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:06.940685 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.909505 21283 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:05:06.941429 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.890674 21064 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:06.930474 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:06.949630 21289 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:06.950850 21192 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.905700 21293 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:05:06.900887 21287 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:06.952679 21291 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:06.924130 21195 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.903990 21277 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:06.915261 21275 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:06.952445 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:06.949759 21150 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.002s
I20260709 13:05:06.904385 21279 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 13:05:06.883339 21285 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.320s
I20260709 13:05:06.982872 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:06.926507 21266 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.985683 21271 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:06.996951 15230 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:06.957808 21151 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:07.002413 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 168857us
I20260709 13:05:06.990823 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:06.991936 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.992007 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:06.943120 21168 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.903326 21270 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:05:07.006206 15162 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:06.983664 21275 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:06.957091 21289 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:06.905184 21175 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:06.985424 21195 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:06.970489 21291 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:06.905076 21290 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:05:06.985236 21277 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:06.904801 21286 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.273s
I20260709 13:05:06.954706 21280 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:06.915711 21269 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:07.019330 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 227344us
I20260709 13:05:07.001013 21080 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:05:07.001732 20604 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.024024 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:07.016800 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:07.010386 21150 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:07.032380 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:07.034421 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:06.983457 21038 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.018654 21277 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.032936 20985 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:07.041824 21038 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:07.042484 20985 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:07.047876 21297 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:05:07.048667 20998 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.922s	user 0.014s	sys 0.000s
I20260709 13:05:07.049533 21292 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:07.050151 21292 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.050622 21292 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.052003 21118 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:07.008764 21261 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.054677 21261 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.055502 15163 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:05:07.055888 15163 pending_rounds.cc:85] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 14
I20260709 13:05:07.056100 15163 pending_rounds.cc:107] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:05:07.056335 15163 raft_consensus.cc:2889] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.012456 21295 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:05:07.030319 21286 consensus_queue.cc:1243] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:07.057435 15086 raft_consensus.cc:2368] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16.
I20260709 13:05:07.031944 21150 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:07.110869 21308 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:07.010016 21287 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:05:07.112771 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.065629 21290 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:07.061574 21194 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:07.107743 21276 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:05:07.025038 21064 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.036620 20604 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:07.151885 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 119581us
I20260709 13:05:07.109726 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:07.107105 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 265761us
W20260709 13:05:07.156378 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.148749 21127 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.002s
I20260709 13:05:07.040192 21302 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 13:05:07.108001 21261 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:05:07.167900 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58248.1us
I20260709 13:05:07.171777 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:07.159304 21194 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.158830 21243 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.057484 21275 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:07.171404 20935 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.762s	user 0.011s	sys 0.000s
I20260709 13:05:07.179620 21194 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:07.176657 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:07.170104 21308 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:07.165992 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:07.222914 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57024.9us
I20260709 13:05:07.190083 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:07.108043 21289 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.226799 21289 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.227218 15841 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:05:07.227622 15841 pending_rounds.cc:85] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 12
I20260709 13:05:07.227849 15841 pending_rounds.cc:107] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:05:07.228070 15841 raft_consensus.cc:2889] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.229031 21261 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:07.229465 21261 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.229790 21261 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:07.231611 21281 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
I20260709 13:05:07.232163 21281 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.233456 15839 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:07.180739 21243 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.235639 21243 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.238346 15085 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:07.195856 21195 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.196998 21150 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.181646 21118 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:07.190461 21259 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.173906 21168 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.173949 21290 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:07.249902 21290 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:07.251087 21290 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.252053 21027 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.221510 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:07.240746 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 64160.5us
I20260709 13:05:07.242054 15847 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:07.240245 21308 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.277814 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:07.228228 21249 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.245962 15082 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:07.278510 15079 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:07.189631 20935 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:07.254034 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:07.280164 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:07.315752 15084 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:07.257781 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.316943 21306 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:07.317273 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.317561 21306 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.318444 15088 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:07.253965 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:07.253772 15085 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:07.232218 21278 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.322417 15842 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:07.311507 21292 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.323933 21278 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.258112 21259 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:07.253644 21184 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.001s
I20260709 13:05:07.327135 21259 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.329315 21241 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:07.327862 21259 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:07.335441 21195 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:07.253073 21150 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.323930 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:07.245254 21267 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:07.338456 21267 raft_consensus.cc:2764] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:05:07.338662 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 84763.4us
I20260709 13:05:07.338805 21303 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:07.339426 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:07.334834 21259 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:07.341111 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.313896 21282 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:07.252804 21277 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.327165 21184 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16.
I20260709 13:05:07.342350 21275 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.319828 21306 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.339547 21303 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:07.340445 21314 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:05:07.341354 21120 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.342991 21259 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.346206 21120 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:07.346274 21168 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:07.342406 21282 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:07.346935 15844 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 16. Ops: []
I20260709 13:05:07.350039 21261 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.349231 21303 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.343806 21277 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.352439 21261 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.232689 21281 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.355168 21281 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.356062 21308 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:07.358301 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 534390us
I20260709 13:05:07.359637 21303 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:07.347067 15228 raft_consensus.cc:2368] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:07.363962 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.364573 15155 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.314333 21240 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:07.374745 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.380154 15159 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:07.246394 21317 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:07.384989 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:07.386560 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:07.388084 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:07.390654 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 131556us
I20260709 13:05:07.392743 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:07.393834 15841 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:07.409754 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:07.412688 15158 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:07.252828 21290 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:07.419680 15155 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:07.429747 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:07.430440 15084 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:07.432375 15079 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:07.434463 15084 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:07.390571 21236 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:07.391942 21281 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:07.404814 21267 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:07.439234 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:07.454577 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.455101 21272 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:07.455567 21272 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:07.455919 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.461654 21249 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.390918 21042 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:05:07.464315 21292 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.465045 21243 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:07.469676 15234 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.470712 15234 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:07.471431 21259 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.473397 21266 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:07.473809 21266 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:07.474622 21266 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.475797 21266 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.456459 21272 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.477273 21272 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.478427 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 2982) took 1053 ms. Trace:
I20260709 13:05:07.478806 15086 rpcz_store.cc:276] 0709 13:05:06.424806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.733931 (+309125us) service_pool.cc:224] Handling call
0709 13:05:07.478242 (+744311us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400349254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:07.478370 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571819776}
I20260709 13:05:07.479527 21194 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:07.482668 21298 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.349s	user 0.003s	sys 0.000s
I20260709 13:05:07.490429 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 2981) took 1059 ms. Trace:
I20260709 13:05:07.489202 15848 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 12. Current term is 14. Ops: [12.10-12.10]
I20260709 13:05:07.456561 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:05:07.494858 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.496335 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:07.496569 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:07.497282 15232 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:05:07.497182 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:07.498070 15237 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.497216 15233 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:07.499655 15082 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:07.490892 15163 rpcz_store.cc:276] 0709 13:05:06.431396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.431593 (+   197us) service_pool.cc:224] Handling call
0709 13:05:06.431867 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.056738 (+624871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.490193 (+433455us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b540033f754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:07.490371 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433404160}
I20260709 13:05:07.523964 15232 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:05:07.524312 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:07.525192 15236 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.508741 15814 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:07.446942 15845 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:05:07.448596 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.536000 15155 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:07.448657 15079 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:07.419530 21261 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:07.546713 15159 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:07.551551 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.552862 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.553823 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.554324 15159 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:07.523957 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:07.555285 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.555799 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:07.548133 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:07.410827 21144 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:05:07.557497 15051 leader_election.cc:400] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 11 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 11. Current term is 12.
I20260709 13:05:07.445060 21236 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:07.571960 15232 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:05:07.572492 15232 pending_rounds.cc:85] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 8
I20260709 13:05:07.572718 15232 pending_rounds.cc:107] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:05:07.573143 15232 raft_consensus.cc:2889] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.573230 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:07.489130 21267 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:07.581581 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 263723us
I20260709 13:05:07.594915 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:07.599759 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.574819 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:07.606400 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.501338 21245 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:07.571709 20870 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:07.482676 21259 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.609344 15155 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.608713 15163 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.621399 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:07.621688 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:07.622042 15079 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.559013 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.549288 21261 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:07.633407 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.576818 21236 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.685916 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.686813 15079 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:07.686810 15161 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:07.688550 15231 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:05:07.691900 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 150174us
I20260709 13:05:07.704959 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:07.625618 21082 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.000s
I20260709 13:05:07.613817 21267 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.557302 21306 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:05:07.749540 15231 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:07.619366 21195 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.616802 21229 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.615067 21245 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:07.723142 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:07.786136 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:07.803426 15161 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:07.678535 20870 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:07.606426 21275 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.149s
I20260709 13:05:07.684981 21261 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.758949 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:07.719477 21286 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.823715 15162 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:07.619719 21308 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:07.827358 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.841144 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 136327us
I20260709 13:05:07.849228 15155 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:07.868206 21321 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:05:07.871073 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:07.882174 15234 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:07.884073 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1322) took 1827 ms. Trace:
I20260709 13:05:07.884958 15234 rpcz_store.cc:276] 0709 13:05:06.056149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.071499 (+ 15350us) service_pool.cc:224] Handling call
0709 13:05:06.071765 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.777218 (+705453us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54000cd8a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:05:07.469301 (+692083us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54000cd8a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f96206f 00007f0d3f9daf60 00007f0d3f9dc7de 00007f0d3f9e6b49 00007f0d3f9e7416 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62
0709 13:05:07.883321 (+414020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.883942 (+   621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175962368}
I20260709 13:05:07.889298 15237 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:07.891638 15159 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:07.900614 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:07.901377 15081 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.827761 21104 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:07.902570 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57832.2us
I20260709 13:05:07.922161 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:07.925107 20995 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:07.925494 20995 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:07.925612 20975 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:07.925905 20995 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.927587 21236 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.929011 21278 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:07.931317 21278 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:07.934374 21289 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:07.941277 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:07.942364 21279 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:07.942860 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2285) took 1007 ms. Trace:
I20260709 13:05:07.943264 15228 rpcz_store.cc:276] 0709 13:05:06.935496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.941162 (+  5666us) service_pool.cc:224] Handling call
0709 13:05:07.942649 (+1001487us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400319a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:07.942795 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931020160}
I20260709 13:05:07.946219 21278 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:07.908456 15236 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:07.910627 21324 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:07.948527 21236 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.842826 21080 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:07.910847 21275 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.949678 21275 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.951871 21275 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.921893 21104 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.954303 21104 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.955498 21104 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:05:07.902787 20981 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:05:07.790818 21082 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.959870 20995 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.960284 21082 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.843451 21286 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.963155 21286 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:07.857885 21008 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.962312 20995 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.964074 21008 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.966612 21008 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:07.964089 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.804962 21195 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:07.975463 21195 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:07.912919 20870 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:07.837365 21194 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:07.815316 21229 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:07.912707 21318 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:08.011703 15162 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 22.
W20260709 13:05:07.997908 21290 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.001s
I20260709 13:05:08.018096 15161 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: [19.11-21.12]
I20260709 13:05:08.034935 15086 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.985309 20995 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.037863 20995 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.040067 20995 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:08.046253 21150 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.003s
I20260709 13:05:08.046934 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.047727 15082 raft_consensus.cc:2376] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13. Re-sending same reply.
I20260709 13:05:08.052263 21240 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.052781 21240 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.053864 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 2987) took 1255 ms. Trace:
I20260709 13:05:08.054240 15158 rpcz_store.cc:276] 0709 13:05:06.798783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.799560 (+   777us) service_pool.cc:224] Handling call
0709 13:05:08.053651 (+1254091us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b540013f6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:08.053779 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339108096}
I20260709 13:05:08.054744 21240 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.056485 20604 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.059413 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1495) took 1069 ms. Trace:
I20260709 13:05:08.059904 15842 rpcz_store.cc:276] 0709 13:05:06.990452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.990574 (+   122us) service_pool.cc:224] Handling call
0709 13:05:08.059176 (+1068602us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54007c4554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:08.059321 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727711104}
W20260709 13:05:08.063040 21266 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.005s
I20260709 13:05:08.063436 21216 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.067103 21229 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.071007 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.074703 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1494) took 1078 ms. Trace:
I20260709 13:05:08.075229 15844 rpcz_store.cc:276] 0709 13:05:06.990125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.990297 (+   172us) service_pool.cc:224] Handling call
0709 13:05:06.990597 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.350944 (+360347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.068133 (+717189us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400852ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:08.068376 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819040128}
I20260709 13:05:08.021101 21308 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:08.082001 15086 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
W20260709 13:05:08.086300 21284 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.264s	user 0.008s	sys 0.011s
I20260709 13:05:08.087752 21271 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:08.091070 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.132143 15082 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:08.133132 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.133422 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.134232 15082 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:08.136708 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:08.138190 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.141078 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.105997 15235 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:08.100087 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.072119 15081 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:08.104210 15231 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:07.971805 21245 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:08.125661 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.103371 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.146214 15234 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:08.151501 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.139045 15082 raft_consensus.cc:2376] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15. Re-sending same reply.
I20260709 13:05:08.174594 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.175959 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.077157 20846 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:05:08.212100 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.213269 15234 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:08.214838 15082 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.215601 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.216503 15081 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:08.139155 20975 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:08.225530 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.139449 21271 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:08.221321 15844 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:05:08.226365 15231 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:08.222448 15847 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:08.223685 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.224460 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.228976 15081 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:08.226009 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:08.225602 20975 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.232970 21008 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:08.233150 20975 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.233528 21008 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:08.234021 21008 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:08.219033 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.224875 21104 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:08.245455 21104 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:08.246440 21104 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.249358 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.249867 21008 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.250805 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.244635 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.252614 21008 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:08.258242 21271 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.261080 20975 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:08.262835 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.263612 20975 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:08.264919 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2280) took 1782 ms. Trace:
W20260709 13:05:08.265080 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.265618 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.265339 15229 rpcz_store.cc:276] 0709 13:05:06.482715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.482836 (+   121us) service_pool.cc:224] Handling call
0709 13:05:06.483167 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.492160 (+  8993us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:07.014960 (+522800us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.166798 (+151838us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.167220 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.167343 (+   123us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:07.167561 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.174345 (+  6784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.174527 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.253648 (+ 79121us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b1800223fc0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:07.867724 (+614076us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b1800223fc0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:08.259095 (+391371us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b1800223fc0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:08.264690 (+  5595us) raft_consensus.cc:1685] finished
0709 13:05:08.264711 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.264861 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34562,"prepare.queue_time_us":109922,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":16015,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":30093,"spinlock_wait_cycles":29806720,"thread_start_us":582968,"threads_started":2,"wal-append.queue_time_us":154802}
I20260709 13:05:08.267390 15232 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:08.267473 21267 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.215411 21279 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:08.268986 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.269300 21279 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.270609 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.272099 21279 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:08.273931 21261 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.275774 15087 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:08.218103 21245 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:08.278790 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1433) took 1058 ms. Trace:
I20260709 13:05:08.279145 15845 rpcz_store.cc:276] 0709 13:05:07.219996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.221219 (+  1223us) service_pool.cc:224] Handling call
0709 13:05:08.278629 (+1057410us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:08.278735 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425542400}
I20260709 13:05:08.283792 15082 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:08.281569 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.285431 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.286043 15843 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:08.287681 15839 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:08.281805 15842 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:08.224464 20976 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.295394 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.296190 15234 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.226490 21230 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.226348 21289 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:08.281229 21064 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.306308 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.292613 20976 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.308434 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.297962 21279 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.299679 20935 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.331528 20935 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:08.331759 21267 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.290946 21269 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.317809 15841 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:08.310426 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1463) took 1950 ms. Trace:
I20260709 13:05:08.335505 15088 rpcz_store.cc:276] 0709 13:05:06.360433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.361932 (+  1499us) service_pool.cc:224] Handling call
0709 13:05:06.362207 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.318847 (+956640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.310129 (+991282us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b540014d054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:08.310354 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2198689664}
I20260709 13:05:08.318723 21231 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:05:08.330499 20976 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:08.346454 15234 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:08.305581 21277 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:05:08.353025 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.014s	sys 0.026s
I20260709 13:05:08.318974 21266 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:08.355439 15085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.003s	sys 0.013s
W20260709 13:05:08.354554 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.014s	sys 0.026s
I20260709 13:05:08.332518 20935 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:08.357452 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.355880 15085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.003s	sys 0.013s
I20260709 13:05:08.352787 21304 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.314203 21230 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.317721 21289 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.353866 21269 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:08.320240 21064 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.365571 15810 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.378043 21309 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.338434 21279 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.382941 15845 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:08.378583 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.359429 21266 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.378683 21309 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:08.388154 20981 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.363605 21304 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:08.408524 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.409066 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.412925 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:08.414183 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.414932 15233 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:08.419229 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.420255 15233 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:08.420596 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.402637 21231 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:08.363433 21291 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.431762 15841 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:08.432500 15841 raft_consensus.cc:606] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:08.421691 20870 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:08.408385 20981 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.449620 20981 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.452976 21304 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.454180 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:08.421799 21229 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:05:08.455245 21266 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.456442 21064 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.458491 21080 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.459105 21080 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.438126 21150 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:05:08.465451 21334 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:08.475793 21291 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:08.482219 21249 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.483057 21249 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:08.486907 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 1.02036e+06us
W20260709 13:05:08.489283 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.491801 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:08.495940 21259 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.496660 21259 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.497073 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:08.497787 15154 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:08.499140 21259 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.499303 21309 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.500375 21259 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.501175 15161 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:08.501345 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.502625 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2362) took 1137 ms. Trace:
I20260709 13:05:08.492643 15843 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:08.505664 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51576.4us
I20260709 13:05:08.503096 15234 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:08.503039 15085 rpcz_store.cc:276] 0709 13:05:07.365477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.365773 (+   296us) service_pool.cc:224] Handling call
0709 13:05:07.403863 (+ 38090us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.421124 (+ 17261us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:07.693765 (+272641us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.361962 (+668197us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.382328 (+ 20366us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.382516 (+   188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:08.382569 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.382669 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.502421 (+119752us) raft_consensus.cc:1685] finished
0709 13:05:08.502443 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.502566 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17955,"prepare.queue_time_us":436107,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":39,"raft.queue_time_us":18480,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":7480,"thread_start_us":780216,"threads_started":2,"wal-append.queue_time_us":601955}
I20260709 13:05:08.508447 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:08.509845 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.510248 15162 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:08.510648 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.511765 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.512624 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.512830 15083 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:08.513283 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.514357 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.515667 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.516476 15233 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:08.519265 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.522471 15086 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:08.523699 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.523836 21229 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:08.496690 21241 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.525113 15161 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:08.561403 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.525650 21150 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:08.526965 21261 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.527459 21064 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.529687 15085 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:08.529842 15155 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:08.532702 21282 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:08.534358 21326 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.431819 21269 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:08.581449 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.430058 20604 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:05:08.445330 21271 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:08.537897 15081 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:08.538509 20870 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:08.538602 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:08.541039 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 53194.2us
I20260709 13:05:08.594594 15083 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:08.542624 21243 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.546399 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.601781 15084 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:08.552326 15157 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:08.554248 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.557682 15154 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:05:08.567188 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.564220 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57188.1us
I20260709 13:05:08.606335 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:08.572520 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.583645 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:08.607635 15841 raft_consensus.cc:2368] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
W20260709 13:05:08.586796 15230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1332) took 1989 ms (client timeout 1707 ms). Trace:
W20260709 13:05:08.608247 15230 rpcz_store.cc:269] 0709 13:05:06.597819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.597998 (+   179us) service_pool.cc:224] Handling call
0709 13:05:08.586539 (+1988541us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54002783d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:08.586706 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879670528}
I20260709 13:05:08.595878 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:08.596580 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.541968 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.545234 21266 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.572907 15162 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:08.613227 15163 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:05:08.561403 20981 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.551468 21229 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:08.590133 21345 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:05:08.614583 15085 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:08.627900 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.613801 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.540315 21259 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.564615 21150 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:08.610317 15086 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16. Re-sending same reply.
W20260709 13:05:08.622731 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 119938us
I20260709 13:05:08.562580 21192 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:05:08.625253 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.615998 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.598491 21271 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:08.581862 21241 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.655092 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:08.656205 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 141915us
I20260709 13:05:08.667913 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:08.616230 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.657441 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.611687 21195 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.660630 15080 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:08.660552 15846 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:08.603624 20916 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.621953 21303 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:08.612910 21306 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.676199 21195 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.677289 21303 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:08.677686 21303 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:08.678042 21303 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:08.678864 21195 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.679540 21195 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:08.680546 20998 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.681030 20916 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.681187 20998 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.682287 21259 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.666218 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.682881 21259 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.683451 21259 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.684196 15127 leader_election.cc:400] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:08.684574 15127 leader_election.cc:403] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:08.685021 21327 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:08.685572 21327 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:08.686021 21347 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:05:08.686415 21327 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.615228 21243 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.689760 21151 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.690284 21151 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.690747 21243 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.691582 15126 leader_election.cc:400] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:08.691968 15126 leader_election.cc:403] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:08.692170 21151 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.693554 21245 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:08.594496 21282 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.696714 21282 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.697723 15846 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:05:08.698856 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43905.8us
I20260709 13:05:08.699115 21240 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:08.700582 21240 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.701242 21240 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:08.587905 21064 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.632207 20604 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.702227 21240 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.702556 20604 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.703258 21064 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.676337 21306 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:08.704267 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 88132.2us
I20260709 13:05:08.594794 21326 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:08.631055 20981 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.706774 20981 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.707617 20981 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:08.707770 21192 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:08.708073 20981 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:08.708253 20995 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:08.708740 20995 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.709143 20995 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.710398 21306 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:08.628996 21229 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:08.668977 21342 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.691489 21243 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:08.623201 21341 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.668311 21144 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:08.720458 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:08.722647 15843 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:08.736394 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.665583 21271 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:08.738643 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.742327 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.742769 15846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:08.745198 15846 raft_consensus.cc:606] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.745594 15846 raft_consensus.cc:614] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:08.746318 15085 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:08.758122 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.765070 15159 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:05:08.765683 15159 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 11
I20260709 13:05:08.765985 15159 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:05:08.766291 15159 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:08.743824 15161 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:08.768915 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.692692 21151 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:05:08.771572 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:08.787117 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.794067 15080 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:08.711829 20604 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.627859 21266 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.671739 21249 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.833703 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:08.708289 21192 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:08.818288 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.818554 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:08.836537 15842 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:08.729712 21070 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.729350 21243 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.727061 21242 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.832533 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.839406 15154 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 13:05:08.849421 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.861335 20935 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.862221 20935 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.866329 20935 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.881582 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.882340 15161 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:08.737285 21064 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.785806 21341 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:08.889367 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:08.896284 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.898089 15087 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:08.901783 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.901849 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:08.902639 15080 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:08.902640 15233 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:08.838981 21282 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:08.738552 21342 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.911477 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:08.912195 15230 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:08.843241 21243 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:08.905164 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:08.909062 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1422) took 1649 ms. Trace:
I20260709 13:05:08.893551 21249 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:08.893573 20935 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.890702 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 348776us
W20260709 13:05:08.924212 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 306320us
I20260709 13:05:08.742388 20870 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.745244 20981 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:08.795243 21150 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.910306 21231 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:08.888444 21242 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.904170 20976 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:05:08.919243 21249 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.915791 15156 rpcz_store.cc:276] 0709 13:05:07.259999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.260602 (+   603us) service_pool.cc:224] Handling call
0709 13:05:07.260952 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.261192 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:07.383955 (+122763us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.050841 (+666886us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b1800467d40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:05:08.398431 (+347590us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.885088 (+486657us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.901369 (+ 16281us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.901428 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:08.901486 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.901548 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.908940 (+  7392us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":270400,"prepare.queue_time_us":331330,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":130,"raft.queue_time_us":901849,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":18177,"spinlock_wait_cycles":21992704,"thread_start_us":639964,"threads_started":3}
I20260709 13:05:08.959779 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:08.894764 21064 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:08.935036 21230 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:08.940641 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:08.953455 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.921772 21144 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:08.936822 20998 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.962977 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:08.963771 21168 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:08.966909 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 353162us
I20260709 13:05:08.915011 21282 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:08.977929 21355 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:08.960162 20894 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:08.983402 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 348439us
I20260709 13:05:08.961409 21361 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.972381 21241 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:08.961992 21249 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.995546 15082 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:08.998867 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:08.999688 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 163961us
I20260709 13:05:08.957163 21231 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:08.926697 20935 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:08.985260 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:08.924656 21192 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:08.975454 21144 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.025782 21118 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:09.058881 21266 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.031463 21361 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.059508 20995 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:09.063024 21291 consensus_meta.cc:287] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.004s
I20260709 13:05:09.052802 21308 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.065785 21290 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:09.067797 21326 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.068724 21243 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:09.074124 21354 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.398s	user 0.004s	sys 0.001s
I20260709 13:05:09.079177 21259 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.080363 21070 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
W20260709 13:05:09.080642 21348 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.391s	user 0.002s	sys 0.000s
I20260709 13:05:09.082034 21342 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.082504 21192 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.083881 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:09.102788 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 197699us
I20260709 13:05:09.003115 21341 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.057834 21241 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.082048 21349 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.411s	user 0.000s	sys 0.002s
I20260709 13:05:09.087630 21118 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:09.147475 21241 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.148375 21118 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.149616 21118 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:09.149988 21241 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.151814 15846 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:09.153040 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.154139 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.155182 21243 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.157238 21242 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:09.157650 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1415) took 2166 ms (client timeout 2080 ms). Trace:
I20260709 13:05:09.113909 21308 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:09.158277 15157 rpcz_store.cc:269] 0709 13:05:06.991171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.991675 (+   504us) service_pool.cc:224] Handling call
0709 13:05:09.157460 (+2165785us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540037e954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:09.157578 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3556389120}
I20260709 13:05:09.160084 21308 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.163863 15162 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:09.164927 15155 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:09.084986 21376 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.121552 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.123579 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.131788 15161 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:09.132504 15842 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:09.110347 21258 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:05:09.134449 21359 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:05:09.133944 21231 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.194192 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:09.194561 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:09.194141 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.195528 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1488) took 1126 ms. Trace:
W20260709 13:05:09.136997 21358 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:05:09.138188 21064 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.119866 21362 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:05:09.120563 21290 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:09.145325 21357 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:05:09.146335 21229 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.103098 21306 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:09.181084 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57622.9us
I20260709 13:05:09.188701 15157 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:09.192659 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.196098 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.134927 21369 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:09.109194 21303 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.203145 15842 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:09.203277 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 204540us
I20260709 13:05:09.230772 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.192134 21008 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.189790 21291 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.234560 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.242852 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.243292 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:09.244182 15229 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:09.247871 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.248595 15846 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:09.227286 21306 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.254074 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.229241 21282 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.255107 15054 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.262723 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.247396 21363 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.244783 21008 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:09.244642 21291 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.219028 21168 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:05:09.239691 21369 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:09.268599 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:09.194093 21376 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.256006 21306 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:09.283660 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.304368 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.277936 21282 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:09.304003 21341 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:09.300927 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 268836us
I20260709 13:05:09.285070 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.256608 21365 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:09.317003 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 58526.8us
I20260709 13:05:09.324960 15846 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:09.325970 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.277897 21008 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:09.330677 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 62181.7us
I20260709 13:05:09.262077 20894 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:09.313344 21366 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:05:09.362465 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.358969 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.366088 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.282001 21291 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:09.366775 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 381621us
I20260709 13:05:09.369812 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.306159 21382 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.363148 15160 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:09.371624 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.315776 21279 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.375833 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:09.372293 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:09.408756 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.354521 21282 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.372745 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:09.543833 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 171149us
W20260709 13:05:09.544379 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 138246us
I20260709 13:05:09.545102 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.546808 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.547540 15085 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.362716 21326 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.421819 15235 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:09.559717 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.423022 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.422885 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.424237 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.427838 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.342667 21341 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:05:09.433710 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.404357 20995 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.405354 21373 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.369406 21383 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:09.376675 21384 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.419560 21144 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:09.433679 21357 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:09.419889 21361 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:09.422319 20935 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.440163 21369 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.440634 21398 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.419575 21393 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:05:09.498857 21387 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:09.500722 21365 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:09.509328 21380 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
W20260709 13:05:09.513638 21390 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:09.509734 21008 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.463271 20894 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.461288 21400 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.518548 21391 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:05:09.636162 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 263935us
I20260709 13:05:09.636827 15159 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:09.516188 21249 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.529500 21290 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.531561 21080 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.619570 21384 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.518330 21376 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.469549 21303 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.536608 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.657426 15160 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:05:09.319015 21367 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:05:09.551057 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.561540 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.661355 15084 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:09.662739 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.562093 15237 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:09.664436 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:09.665704 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.666363 15237 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:09.667290 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.667899 15237 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
W20260709 13:05:09.561465 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133683us
I20260709 13:05:09.563998 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.673018 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.567483 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.316689 21306 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.569931 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.573803 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.583659 21392 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:09.543340 21282 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.585556 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.587384 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1427) took 1093 ms. Trace:
I20260709 13:05:09.685572 15158 rpcz_store.cc:276] 0709 13:05:08.493898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.499955 (+  6057us) service_pool.cc:224] Handling call
0709 13:05:08.500284 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.712002 (+211718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.712044 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.712120 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:08.712166 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.587111 (+874945us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400187ed4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:09.587165 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.587321 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486400896}
I20260709 13:05:09.593293 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:09.687424 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 94244.4us
W20260709 13:05:09.594898 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 49875.9us
I20260709 13:05:09.602370 15085 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:09.558775 21064 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:09.559095 21281 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:09.616714 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 246955us
I20260709 13:05:09.694648 15163 raft_consensus.cc:2368] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:09.694727 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:09.596365 21266 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.703948 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.596825 21261 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.605373 21269 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.597879 21304 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.565863 21308 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:05:09.462842 21229 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:09.642005 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:09.732399 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90484.8us
I20260709 13:05:09.570225 21326 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.597414 21373 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.618151 21398 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.645466 21395 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.646649 21279 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.647713 21104 raft_consensus.cc:993] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: : Instructing follower 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:05:09.660101 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.661428 15231 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:09.749578 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.561465 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.671648 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.747196 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:09.750432 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:05:09.750586 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79043.3us
I20260709 13:05:09.751253 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.751349 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.740095 21402 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:09.752346 21327 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:09.752558 21402 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.752758 21327 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.752995 21402 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.753219 21327 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.679783 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.756443 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.757032 15237 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:09.757542 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.757769 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.758299 15237 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:09.758490 21326 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.759020 15231 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:09.673292 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.681931 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:09.663319 15084 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:09.760943 21373 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.761492 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.762204 15084 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:09.763051 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.763984 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.678386 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.692531 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:09.640159 21400 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:09.765413 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72963.3us
I20260709 13:05:09.766959 21306 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.723811 21266 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.685135 15230 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:05:09.769485 21266 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.769340 21327 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.769959 15230 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:09.770340 21280 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.744253 21080 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:09.770879 20935 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.704650 15158 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:09.771319 20935 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.771709 20935 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.638442 21388 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.772608 21388 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.732194 21269 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.773463 21308 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:09.774567 21383 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:09.683653 21271 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.000s
I20260709 13:05:09.774896 21383 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:09.743393 21395 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:09.771322 21118 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.775660 21395 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.775997 21118 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.777424 15083 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:09.778143 21118 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.771925 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:09.779938 21356 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.097s	user 0.000s	sys 0.005s
I20260709 13:05:09.775173 21271 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.781876 21249 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.782030 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.782564 21271 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.782650 21301 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:09.780277 20935 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.783172 21301 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:09.774892 21388 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.783893 21398 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.775177 21269 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.693599 21290 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:05:09.661206 21384 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.786406 15847 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:09.787109 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1526) took 1182 ms. Trace:
I20260709 13:05:09.682125 21303 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:09.696063 21282 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.742664 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.705439 21144 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:09.696870 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:09.779850 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:09.681696 21376 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.795569 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122336us
I20260709 13:05:09.730242 15087 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:05:09.788820 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 110563us
I20260709 13:05:09.797251 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.797935 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.798061 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.789515 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.787869 15128 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.724766 21304 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.791298 15081 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.791234 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.787467 15841 rpcz_store.cc:276] 0709 13:05:08.605126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.675732 (+ 70606us) service_pool.cc:224] Handling call
0709 13:05:08.676141 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.697851 (+ 21710us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:08.697879 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.698330 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.966225 (+267895us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:08.967960 (+  1735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.968039 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.786905 (+818866us) raft_consensus.cc:1685] finished
0709 13:05:09.786925 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.787052 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21298,"raft.queue_time_us":24513,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":267445,"threads_started":1,"wal-append.queue_time_us":356695}
I20260709 13:05:09.800977 15084 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:09.794494 15160 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:09.809996 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.813246 15081 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:09.814462 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.815153 15081 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:09.818024 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:09.820044 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.812992 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:09.785915 21241 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:09.810653 15084 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:09.772437 21291 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.820963 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.842180 15235 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:09.844619 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:09.848111 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100988us
I20260709 13:05:09.809142 21362 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:09.846292 21266 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.841763 21241 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.853223 21008 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.743924 21365 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:05:09.819104 21269 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.854270 21269 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.855258 21382 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:09.823985 21249 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.855019 21383 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.855959 21249 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.856336 21383 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.856420 21249 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:09.856810 21269 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.857406 21269 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:09.858038 21269 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:09.843266 21261 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.843225 21394 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:09.846437 21259 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:09.862079 20935 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.862517 21259 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.862931 21259 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.863736 21271 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:09.854959 21362 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.842933 21373 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.784098 21301 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.864624 21362 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.866605 21384 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.867223 21377 raft_consensus.cc:1081] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Signalling peer 12daa67457954a0cab27e32bf85fa115 to start an election
I20260709 13:05:09.867511 21400 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.848199 21243 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
W20260709 13:05:09.864148 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.853667 21241 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.870911 21395 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.788800 21279 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:09.873338 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.875053 21279 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.202250 15079 rpcz_store.cc:276] 0709 13:05:08.069556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.449357 (+379801us) service_pool.cc:224] Handling call
0709 13:05:08.449669 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.449884 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:05:08.965157 (+515273us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.193542 (+228385us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.193616 (+    74us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:09.193681 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.193776 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.195467 (+  1691us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178565,"prepare.queue_time_us":473930,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":1357,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11226,"thread_start_us":564133,"threads_started":2}
I20260709 13:05:09.876221 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:09.876763 21363 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:05:09.877256 21363 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:09.878345 21384 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:09.878401 21241 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.810103 21303 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:09.878991 15232 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:09.879040 21384 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.880955 21230 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.881371 21230 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.881716 21230 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.844638 21118 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:09.881939 21384 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:09.882405 21118 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.882817 21118 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.884527 15236 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:09.886765 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:09.888921 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.830663 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:09.890223 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59656.8us
I20260709 13:05:09.890547 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:09.794374 21290 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:09.792006 21229 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:09.810730 21304 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:09.900484 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.901758 15087 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:09.764472 21402 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.819034 21393 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.811787 21144 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.915526 15841 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:09.892911 21275 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:09.932763 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.933607 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:09.899724 21261 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:09.935310 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.899187 21236 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:09.943370 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45045.9us
I20260709 13:05:09.924773 21144 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.928401 20981 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.927955 21393 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.956652 20981 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.900857 20935 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:05:09.898332 21384 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:09.928323 21406 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.821468 21376 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:09.932368 21271 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:09.941110 15087 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:09.945775 21236 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:09.905706 21229 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:09.981355 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 183345us
I20260709 13:05:09.981707 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:09.988724 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 144849us
I20260709 13:05:09.978646 21236 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:09.963156 21373 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.990288 15813 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.581s (SENT) [suppressed 22 similar messages]
I20260709 13:05:09.991174 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:09.963235 21395 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:09.994067 15157 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:09.967855 21406 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.996380 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.997375 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.001505 15839 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:09.975818 21388 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.996680 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:10.006724 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.003208 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:10.009259 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.010840 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.012197 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.996066 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:09.989610 21261 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:10.020462 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:10.025218 15839 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.022454 15087 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.020314 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.026202 15843 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:10.027760 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:09.964463 21405 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:10.035091 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:10.011499 21243 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:10.005610 21406 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.053339 15128 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:10.049329 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.024219 21384 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:10.055030 21384 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.055543 21384 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:10.057267 15842 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:10.059669 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.024312 21363 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.061977 15087 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:10.069170 15052 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:10.070290 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:10.071071 15087 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:10.077733 15839 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:10.067538 21409 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:10.053884 21243 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.998735 21373 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:10.047863 21261 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.020905 21388 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:10.095507 15087 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:10.099045 21326 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:10.069298 21393 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:10.072619 21271 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:10.053722 21416 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:10.109134 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:10.940718 15087 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:10.127331 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2294) took 1531 ms. Trace:
I20260709 13:05:10.114755 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:10.126783 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:10.169763 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1481) took 2172 ms. Trace:
W20260709 13:05:10.732545 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.015s	sys 0.012s
W20260709 13:05:10.944234 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.017s	sys 0.012s
I20260709 13:05:10.943113 15233 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:10.945329 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2311) took 1122 ms. Trace:
I20260709 13:05:10.210485 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:05:10.945883 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.945768 15233 rpcz_store.cc:276] 0709 13:05:09.822774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.114486 (+291712us) service_pool.cc:224] Handling call
0709 13:05:10.945268 (+830782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.942546 15229 rpcz_store.cc:276] 0709 13:05:08.595540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.596958 (+  1418us) service_pool.cc:224] Handling call
0709 13:05:08.597373 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.250169 (+652796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.127036 (+876867us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b540005bc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:10.127262 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487102720}
W20260709 13:05:10.313268 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.264258 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:10.953799 15844 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.141304 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2575) took 1424 ms. Trace:
I20260709 13:05:10.955399 15162 rpcz_store.cc:276] 0709 13:05:08.717329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.717502 (+   173us) service_pool.cc:224] Handling call
0709 13:05:09.163408 (+445906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.164485 (+  1077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.141074 (+976589us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540037e950. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:10.141231 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964634880}
I20260709 13:05:10.958161 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:10.959070 15839 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:10.254858 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:10.258829 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:10.939298 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:10.965196 15235 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:10.938807 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3004) took 1267 ms. Trace:
I20260709 13:05:10.966645 15159 rpcz_store.cc:276] 0709 13:05:09.670977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.793126 (+122149us) service_pool.cc:224] Handling call
0709 13:05:09.795898 (+  2772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.841564 (+ 45666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.841613 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.848105 (+  6492us) log.cc:844] Serialized 26 byte log entry
0709 13:05:10.005075 (+156970us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:10.005131 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.005226 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.147302 (+142076us) raft_consensus.cc:1685] finished
0709 13:05:10.147325 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.938710 (+791385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15902,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2475,"thread_start_us":79461,"threads_started":1,"wal-append.queue_time_us":148772}
W20260709 13:05:10.940452 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.002s	sys 0.008s
W20260709 13:05:10.969085 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.002s	sys 0.008s
W20260709 13:05:10.952641 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.943732 15088 rpcz_store.cc:276] 0709 13:05:07.997291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.337216 (+339925us) service_pool.cc:224] Handling call
0709 13:05:08.353401 (+ 16185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.396459 (+ 43058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.396493 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.398129 (+  1636us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.429818 (+ 31689us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:08.429875 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.169263 (+1739388us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b540034a654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:10.169331 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.169398 (+    67us) raft_consensus.cc:1685] finished
0709 13:05:10.169417 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.169691 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1045,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2634,"spinlock_wait_cycles":98409472,"thread_start_us":31149,"threads_started":1,"wal-append.queue_time_us":234400,"wal-append.run_cpu_time_us":1738,"wal-append.run_wall_time_us":1055998}
I20260709 13:05:10.079885 20935 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:10.079214 20981 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:10.079965 21290 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:10.091547 21291 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:10.982780 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:10.123138 21412 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:05:10.843161 21144 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.000s
W20260709 13:05:10.256721 21414 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:05:11.002671 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.989358 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:11.038143 15236 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:10.214545 20916 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:10.763283 21249 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.906s	user 0.010s	sys 0.000s
W20260709 13:05:10.843607 21386 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.322s	user 0.002s	sys 0.003s
W20260709 13:05:10.145761 21407 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:05:10.125622 21401 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.499s	user 0.000s	sys 0.002s
W20260709 13:05:10.595758 21118 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.010s
I20260709 13:05:10.083863 21042 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:10.741734 21259 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.873s	user 0.014s	sys 0.007s
W20260709 13:05:10.699874 21282 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.881s	user 0.016s	sys 0.008s
W20260709 13:05:11.044572 21428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.013s	sys 0.010s
W20260709 13:05:10.541092 21304 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.617s	user 0.007s	sys 0.004s
I20260709 13:05:10.093278 21243 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:10.794662 21406 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.722s	user 0.005s	sys 0.005s
W20260709 13:05:10.803247 21383 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.946s	user 0.006s	sys 0.005s
W20260709 13:05:10.843422 21230 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.961s	user 0.010s	sys 0.009s
W20260709 13:05:10.177101 21379 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.898s	user 0.002s	sys 0.000s
W20260709 13:05:11.018646 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1436) took 2178 ms (client timeout 1567 ms). Trace:
I20260709 13:05:11.018477 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:10.744668 21384 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.001s
I20260709 13:05:10.983409 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:11.070457 15161 rpcz_store.cc:269] 0709 13:05:08.840508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.883158 (+ 42650us) service_pool.cc:224] Handling call
0709 13:05:11.018430 (+2135272us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54001c5cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.018564 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527256832}
I20260709 13:05:11.071364 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2576) took 2341 ms. Trace:
I20260709 13:05:11.020908 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2594) took 1029 ms. Trace:
I20260709 13:05:11.071861 15229 rpcz_store.cc:276] 0709 13:05:09.991122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.982523 (+991401us) service_pool.cc:224] Handling call
0709 13:05:11.020814 (+ 38291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.071739 15154 rpcz_store.cc:276] 0709 13:05:08.717897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.718046 (+   149us) service_pool.cc:224] Handling call
0709 13:05:08.718493 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.847687 (+129194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.058747 (+2211060us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540013f6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.059026 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277257088}
I20260709 13:05:11.021425 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:11.024430 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:10.132210 21241 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:10.942108 21416 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.940002 21400 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.025228 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:10.135092 21342 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:10.131692 21275 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:10.079777 21363 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:10.168589 21266 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.131227 21229 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:10.982467 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:10.174623 21373 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:10.122681 21289 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:10.168593 21376 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.040705 15844 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:10.132540 21402 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.106448 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2310) took 1284 ms. Trace:
W20260709 13:05:11.026062 20870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.015s	sys 0.037s
I20260709 13:05:10.124822 21261 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:10.132740 21236 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:10.981992 21326 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:10.737726 21362 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.868s	user 0.006s	sys 0.006s
W20260709 13:05:11.049700 21194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.021s	sys 0.050s
I20260709 13:05:11.105850 21434 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.020555 21290 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.067888 21433 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.054368 21428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.014s	sys 0.011s
I20260709 13:05:11.072937 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.074936 15236 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:11.040674 21144 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.050384 21042 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:11.078136 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2599) took 1024 ms. Trace:
I20260709 13:05:11.152575 15235 rpcz_store.cc:276] 0709 13:05:10.053278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.989989 (+936711us) service_pool.cc:224] Handling call
0709 13:05:11.078064 (+ 88075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.079349 15848 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:11.044818 21249 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.050936 21118 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.053709 21259 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.077149 21384 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.055167 21282 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.080251 21294 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:11.067226 21304 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.077605 21416 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:11.066946 21406 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:10.311883 21341 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.086032 21342 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:11.074990 21230 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:11.110909 20870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.016s	sys 0.037s
I20260709 13:05:11.111889 21439 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.079831 21400 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.038321 21291 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:11.131347 21194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.021s	sys 0.050s
I20260709 13:05:11.096282 21275 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.146082 21402 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:11.157845 21042 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:11.131682 21362 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.037454 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:11.185258 21342 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.148641 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:11.084353 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2307) took 1407 ms. Trace:
I20260709 13:05:11.109174 15233 rpcz_store.cc:276] 0709 13:05:09.822518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.982134 (+1159616us) service_pool.cc:224] Handling call
0709 13:05:11.106364 (+124230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.146862 15234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1355) took 2728 ms (client timeout 1581 ms). Trace:
I20260709 13:05:11.148656 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2312) took 1256 ms. Trace:
I20260709 13:05:11.154218 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.149742 15840 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:11.178653 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:11.136591 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.143808 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.218379 21275 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.068619 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:11.075871 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1451) took 1075 ms. Trace:
I20260709 13:05:11.218009 21384 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.186012 21236 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.236788 21388 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.186904 21261 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:11.341563 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 301612us
I20260709 13:05:11.312119 21144 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:11.307735 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 336620us
I20260709 13:05:11.070704 15084 raft_consensus.cc:2376] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13. Re-sending same reply.
I20260709 13:05:11.218952 21402 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.182760 21294 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.370821 21301 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:11.314297 21249 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.373262 21400 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.373793 21402 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.190117 21291 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:11.375504 15084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1514) took 1247 ms (client timeout 1581 ms). Trace:
W20260709 13:05:11.376123 15084 rpcz_store.cc:269] 0709 13:05:10.127835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.980897 (+853062us) service_pool.cc:224] Handling call
0709 13:05:11.375203 (+394306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.378180 21291 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.378499 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:11.277251 15087 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:11.184928 21427 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.927s	user 0.002s	sys 0.011s
I20260709 13:05:11.308808 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.380088 15088 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:11.145430 21434 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.437885 21434 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.188056 21439 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.333787 15228 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:11.439925 15228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2589) took 1764 ms (client timeout 1785 ms). Trace:
W20260709 13:05:11.440366 15228 rpcz_store.cc:269] 0709 13:05:09.675874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.751130 (+ 75256us) service_pool.cc:224] Handling call
0709 13:05:11.439641 (+1688511us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000582d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.439828 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3636197760}
I20260709 13:05:11.441893 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.443881 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.103825 21266 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:11.445869 21342 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.275739 15080 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:11.447798 15080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2382) took 1681 ms (client timeout 2174 ms). Trace:
W20260709 13:05:11.448199 15080 rpcz_store.cc:269] 0709 13:05:09.766119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.126471 (+360352us) service_pool.cc:224] Handling call
0709 13:05:11.447580 (+1321109us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400190354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.447716 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762571520}
I20260709 13:05:11.324050 21118 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.343330 15843 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:11.150590 21433 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:11.380559 21424 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:05:11.307894 15079 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:11.364100 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:11.403499 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2385) took 1209 ms. Trace:
I20260709 13:05:11.342587 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1504) took 2144 ms. Trace:
I20260709 13:05:11.344864 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1514) took 3125 ms. Trace:
I20260709 13:05:11.358106 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.361238 21269 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.184769 21341 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.415313 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.460060 15088 rpcz_store.cc:276] 0709 13:05:10.193623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.993767 (+800144us) service_pool.cc:224] Handling call
0709 13:05:11.403331 (+409564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.415817 15236 rpcz_store.cc:276] 0709 13:05:09.891991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.989012 (+1097021us) service_pool.cc:224] Handling call
0709 13:05:11.148599 (+159587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.184448 21406 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.328023 21150 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:11.416990 15162 rpcz_store.cc:276] 0709 13:05:10.000720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.989865 (+989145us) service_pool.cc:224] Handling call
0709 13:05:11.075819 (+ 85954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.418244 15842 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:11.418841 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2597) took 1425 ms. Trace:
W20260709 13:05:11.380155 15234 rpcz_store.cc:269] 0709 13:05:08.411938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.412569 (+   631us) service_pool.cc:224] Handling call
0709 13:05:11.140402 (+2727833us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b5400295c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.140540 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205114240}
I20260709 13:05:11.421334 21327 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.185348 21289 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.418854 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2577) took 2699 ms. Trace:
W20260709 13:05:11.422127 15163 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2580) took 2400 ms (client timeout 1708 ms). Trace:
I20260709 13:05:11.460417 15847 rpcz_store.cc:276] 0709 13:05:09.198219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.200281 (+  2062us) service_pool.cc:224] Handling call
0709 13:05:09.200647 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.793350 (+592703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.341721 (+1548371us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54006dc254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.342294 (+   573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346645376}
I20260709 13:05:11.415959 15230 rpcz_store.cc:276] 0709 13:05:09.677103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.770765 (+ 93662us) service_pool.cc:224] Handling call
0709 13:05:09.771073 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.879683 (+108610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.879718 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.879761 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:09.879796 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.084093 (+1204297us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400176454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.084143 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.084293 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249364736}
I20260709 13:05:11.424244 15084 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:11.331408 21192 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:11.189999 21259 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:11.430373 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55824us
W20260709 13:05:11.240268 21430 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:05:11.304428 15846 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:11.323923 21282 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.420522 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1435) took 2586 ms. Trace:
I20260709 13:05:11.465911 15229 rpcz_store.cc:276] 0709 13:05:09.993189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.072667 (+1079478us) service_pool.cc:224] Handling call
0709 13:05:11.418548 (+345881us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.467187 15163 rpcz_store.cc:269] 0709 13:05:09.021850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.083621 (+ 61771us) service_pool.cc:224] Handling call
0709 13:05:11.421528 (+2337907us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54000857d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.421808 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299442944}
W20260709 13:05:11.416064 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1503) took 1755 ms (client timeout 1640 ms). Trace:
I20260709 13:05:11.465139 15081 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:11.244061 21437 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:11.460690 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.425546 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.467543 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:11.460996 15848 rpcz_store.cc:276] 0709 13:05:08.218992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.219511 (+   519us) service_pool.cc:224] Handling call
0709 13:05:08.219863 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.153511 (+2933648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.344145 (+190634us) spinlock_profiling.cc:243] Waited 2.86 s on lock 0x7b54002a1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.344619 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6575724416}
I20260709 13:05:11.307776 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:11.464998 21275 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.475140 15841 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:11.477964 15086 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:11.478721 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:11.239699 21326 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.467090 15155 rpcz_store.cc:276] 0709 13:05:08.719021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.719156 (+   135us) service_pool.cc:224] Handling call
0709 13:05:08.772370 (+ 53214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.165427 (+393057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.418187 (+2252760us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54003e0654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.418590 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902174080}
I20260709 13:05:11.478721 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3023) took 1465 ms. Trace:
W20260709 13:05:11.471755 15083 rpcz_store.cc:269] 0709 13:05:09.626049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.672763 (+ 46714us) service_pool.cc:224] Handling call
0709 13:05:11.380660 (+1707897us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000988d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.380895 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238664064}
I20260709 13:05:11.186645 21230 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.470955 15156 rpcz_store.cc:276] 0709 13:05:08.833960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.973599 (+139639us) service_pool.cc:224] Handling call
0709 13:05:08.973911 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.108787 (+134876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.108820 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.108884 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:09.108918 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.419769 (+2310851us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54001c6bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.419876 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.420268 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309757696}
I20260709 13:05:11.483142 15086 rpcz_store.cc:276] 0709 13:05:10.013477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.068341 (+1054864us) service_pool.cc:224] Handling call
0709 13:05:11.478623 (+410282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.467803 21439 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.489482 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.491287 15841 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:11.491920 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:11.492285 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.494582 21433 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.487634 15235 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:11.491007 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:11.503916 15228 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:11.501520 15082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2372) took 2671 ms (client timeout 2815 ms). Trace:
I20260709 13:05:11.505695 21289 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.507259 21303 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.508001 21266 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.512336 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2300) took 2086 ms. Trace:
I20260709 13:05:11.512781 15237 rpcz_store.cc:276] 0709 13:05:09.425348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.759701 (+334353us) service_pool.cc:224] Handling call
0709 13:05:09.765164 (+  5463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.794739 (+ 29575us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:10.027018 (+232279us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.103036 (+ 76018us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.103669 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:05:10.103830 (+   161us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:10.103879 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.337658 (+1233779us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b540043d354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.337752 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.511716 (+173964us) raft_consensus.cc:1685] finished
0709 13:05:11.511839 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.512121 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16635,"prepare.queue_time_us":169844,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":7970,"raft.queue_time_us":12123,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":12208,"spinlock_wait_cycles":39976192,"thread_start_us":198081,"threads_started":2,"wal-append.queue_time_us":1010999}
I20260709 13:05:11.515543 15237 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:11.516919 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2577) took 2097 ms. Trace:
W20260709 13:05:11.517192 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 209938us
I20260709 13:05:11.517422 15232 rpcz_store.cc:276] 0709 13:05:09.419280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.419402 (+   122us) service_pool.cc:224] Handling call
0709 13:05:09.419711 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.879578 (+459867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.516354 (+1636776us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b540043e754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:11.516670 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055785216}
I20260709 13:05:11.518118 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:11.519418 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:11.520148 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2309) took 1842 ms. Trace:
I20260709 13:05:11.520468 21393 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:11.238389 21290 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.521195 15839 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:11.521894 21308 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.522624 21308 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:11.523859 21454 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:11.521023 21393 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.496701 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1449) took 1512 ms. Trace:
I20260709 13:05:11.524241 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2595) took 1512 ms. Trace:
I20260709 13:05:11.528940 15236 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:05:11.181041 21243 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.557591 21449 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.558514 21243 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.558349 21449 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.561775 21308 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.565481 21449 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.565955 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:11.565850 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:11.567019 15234 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:11.567106 15080 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:11.567641 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3015) took 1799 ms. Trace:
I20260709 13:05:11.567802 15084 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:11.567992 15080 rpcz_store.cc:276] 0709 13:05:09.768408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.485389 (+1716981us) service_pool.cc:224] Handling call
0709 13:05:11.567568 (+ 82179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.572805 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3018) took 1750 ms. Trace:
I20260709 13:05:11.573040 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.573505 15084 rpcz_store.cc:276] 0709 13:05:09.818507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.377906 (+1559399us) service_pool.cc:224] Handling call
0709 13:05:11.568516 (+190610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.524766 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:11.574796 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:11.575831 15084 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:11.576206 15230 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:11.577997 15228 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:11.578822 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2593) took 1756 ms. Trace:
I20260709 13:05:11.579196 15228 rpcz_store.cc:276] 0709 13:05:09.822221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.577001 (+1754780us) service_pool.cc:224] Handling call
0709 13:05:11.577557 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.578503 (+   946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.578729 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.580222 21294 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.580888 21261 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.581678 21261 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.582832 21341 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.532657 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1497) took 1931 ms. Trace:
W20260709 13:05:11.505401 15082 rpcz_store.cc:269] 0709 13:05:08.830041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.832173 (+  2132us) service_pool.cc:224] Handling call
0709 13:05:11.501146 (+2668973us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000dbbd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:11.501325 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340753792}
I20260709 13:05:11.533748 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:11.534400 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.534324 21404 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:11.535046 21275 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.529932 21393 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:11.535893 21363 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.536916 21383 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.537421 21192 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.508598 21289 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:11.536512 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1448) took 1399 ms. Trace:
W20260709 13:05:11.541312 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.003s
I20260709 13:05:11.541579 15843 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:11.541810 15233 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:11.527997 21269 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.543042 21362 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.544051 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:11.546828 21230 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.548964 15128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.007s
I20260709 13:05:11.528499 15235 rpcz_store.cc:276] 0709 13:05:09.991403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.357315 (+1365912us) service_pool.cc:224] Handling call
0709 13:05:11.503369 (+146054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.553588 15083 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:11.555612 21448 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.520696 15232 rpcz_store.cc:276] 0709 13:05:09.677658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.518672 (+1841014us) service_pool.cc:224] Handling call
0709 13:05:11.519329 (+   657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.519718 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.519751 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.519818 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:11.519874 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.519935 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.520068 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.586524 15084 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:11.584156 15086 rpcz_store.cc:276] 0709 13:05:09.571072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.501079 (+1930007us) service_pool.cc:224] Handling call
0709 13:05:11.501588 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.501983 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.502011 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.502078 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:11.502121 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.502185 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.502300 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.587309 15236 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:11.520505 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:11.594179 21383 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.590451 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:11.595224 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51474.7us
W20260709 13:05:11.597628 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.003s	sys 0.004s
I20260709 13:05:11.496766 21304 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.612206 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1499) took 2004 ms. Trace:
I20260709 13:05:11.612668 15084 rpcz_store.cc:276] 0709 13:05:09.594359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.576719 (+1982360us) service_pool.cc:224] Handling call
0709 13:05:11.580529 (+  3810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.598334 (+ 17805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.598540 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.602476 21448 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.587055 21404 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.564402 20976 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:05:11.607961 21281 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:11.600945 15128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.008s
I20260709 13:05:11.609016 21282 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.609695 21290 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.610612 21275 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.597934 21303 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:11.611639 15082 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:11.599144 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:11.528352 15158 rpcz_store.cc:276] 0709 13:05:09.983703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.983949 (+   246us) service_pool.cc:224] Handling call
0709 13:05:09.984333 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.984809 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.984871 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.985446 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:05:10.883854 (+898408us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:11.302689 (+418835us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.303788 (+  1099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.304006 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.496120 (+192114us) raft_consensus.cc:1685] finished
0709 13:05:11.496150 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.496411 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":263264,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2943,"raft.queue_time_us":6875,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":7037,"thread_start_us":298696,"threads_started":2,"wal-append.queue_time_us":1079900}
I20260709 13:05:11.617350 21266 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.591838 15160 rpcz_store.cc:276] 0709 13:05:09.983221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.983368 (+   147us) service_pool.cc:224] Handling call
0709 13:05:09.995010 (+ 11642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.995219 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:10.054557 (+ 59338us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.336068 (+1281511us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18003d1140. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a
0709 13:05:11.359251 (+ 23183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.359360 (+   109us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:11.359489 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.359618 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.382652 (+ 23034us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72809,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":21823,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":227,"spinlock_wait_cycles":26833664,"thread_start_us":70576,"threads_started":2,"wal-append.queue_time_us":68784,"wal-append.run_cpu_time_us":1236,"wal-append.run_wall_time_us":1001264}
I20260709 13:05:11.604200 21192 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:11.622848 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1498) took 2034 ms. Trace:
I20260709 13:05:11.644323 15083 rpcz_store.cc:276] 0709 13:05:09.577307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.529047 (+1951740us) service_pool.cc:224] Handling call
0709 13:05:11.553286 (+ 24239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.600540 (+ 47254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.611666 (+ 11126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.631183 15229 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:11.642388 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:11.647677 21261 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.628446 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:11.650545 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.648510 15082 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:11.652693 21282 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.653918 21304 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:11.654479 21304 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:11.655539 21282 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.656140 21383 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.655612 21304 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.656872 15086 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:11.657577 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1500) took 2054 ms. Trace:
I20260709 13:05:11.624202 21404 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:11.657989 15086 rpcz_store.cc:276] 0709 13:05:09.603298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.628666 (+2025368us) service_pool.cc:224] Handling call
0709 13:05:11.656635 (+ 27969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.657302 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.657491 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.653153 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1504) took 2026 ms. Trace:
I20260709 13:05:11.659619 15082 rpcz_store.cc:276] 0709 13:05:09.627167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.647807 (+2020640us) service_pool.cc:224] Handling call
0709 13:05:11.648240 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.652892 (+  4652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.653080 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.653189 21269 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.643736 21308 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:11.664475 21436 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:11.674877 21308 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:11.675495 21308 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:11.675280 21436 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.676182 21450 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.626573 21281 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:11.666280 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.007s	sys 0.002s
I20260709 13:05:11.645604 21290 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:11.673202 21373 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.621726 21230 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.199813 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.762820 15081 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:12.200379 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:11.739080 15080 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.981238 15158 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:12.238691 21450 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:11.809016 21269 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.241974 21269 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.130913 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1455) took 1121 ms. Trace:
I20260709 13:05:12.211565 20976 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:05:11.691956 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:11.803303 20981 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.246109 20981 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.234853 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.247547 15848 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:12.249064 21266 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:05:12.250039 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 558240us
I20260709 13:05:12.250931 15163 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:05:12.243278 15081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1510) took 2251 ms (client timeout 2227 ms). Trace:
I20260709 13:05:12.245007 20870 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:11.677345 15085 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
W20260709 13:05:12.253539 15085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2383) took 2487 ms (client timeout 2191 ms). Trace:
I20260709 13:05:12.253159 20870 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:12.253933 15085 rpcz_store.cc:269] 0709 13:05:09.766380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.616397 (+1850017us) service_pool.cc:224] Handling call
0709 13:05:12.253467 (+637070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.254724 21400 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.805310 21393 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.255509 21400 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.256542 20870 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.257445 21393 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.257933 21400 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:12.258085 15081 rpcz_store.cc:269] 0709 13:05:09.992211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.363260 (+1371049us) service_pool.cc:224] Handling call
0709 13:05:12.243184 (+879924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.820106 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1450) took 1685 ms. Trace:
W20260709 13:05:12.259799 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.252306 21402 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:12.259213 15159 rpcz_store.cc:276] 0709 13:05:09.985970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.992134 (+1006164us) service_pool.cc:224] Handling call
0709 13:05:10.992557 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.595805 (+603248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.601822 (+  6017us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.622166 (+ 20344us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:11.622208 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.644611 (+ 22403us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.671241 (+ 26630us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119470,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":19341,"thread_start_us":484501,"threads_started":1}
I20260709 13:05:12.260597 21269 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.915778 15237 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:12.260557 21402 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.261765 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2308) took 2584 ms. Trace:
I20260709 13:05:12.248567 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.262149 15237 rpcz_store.cc:276] 0709 13:05:09.677387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.514934 (+1837547us) service_pool.cc:224] Handling call
0709 13:05:11.515374 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.261466 (+746092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.261694 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.263005 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:12.263195 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.006s	sys 0.005s
I20260709 13:05:12.263082 21402 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:11.947469 21301 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.003s
I20260709 13:05:11.739498 21080 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:11.818934 21308 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.293071 21080 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.293653 21281 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.294847 21448 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:12.294962 21281 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:11.739602 21192 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.295477 21080 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.296017 15086 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:12.295924 21192 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.296689 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1506) took 2666 ms. Trace:
I20260709 13:05:11.675275 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:12.176017 21042 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.000s
W20260709 13:05:12.267261 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.006s	sys 0.006s
I20260709 13:05:11.678390 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:12.207413 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:12.325253 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.326529 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.327124 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:11.690022 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:12.327769 15084 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:12.328150 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 638381us
I20260709 13:05:12.268573 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.264418 21373 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.330492 21230 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.330924 21438 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:12.331489 21438 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:12.331574 21373 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.244261 15161 rpcz_store.cc:276] 0709 13:05:11.008495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.071645 (+ 63150us) service_pool.cc:224] Handling call
0709 13:05:11.080054 (+  8409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.080259 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:11.556991 (+476732us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.770508 (+213517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.127914 (+357406us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:12.128291 (+   377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.128661 (+   370us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.129679 (+  1018us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3296,"prepare.queue_time_us":108094,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":25810,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"thread_start_us":548946,"threads_started":2,"wal-append.queue_time_us":126117}
I20260709 13:05:12.332454 21438 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.333711 15161 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:12.297075 15086 rpcz_store.cc:276] 0709 13:05:09.630473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.658982 (+2028509us) service_pool.cc:224] Handling call
0709 13:05:12.295636 (+636654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.296426 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.296618 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.336117 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.337545 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.338127 15086 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.339447 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2376) took 2590 ms. Trace:
I20260709 13:05:12.339896 15081 rpcz_store.cc:276] 0709 13:05:09.748749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.259025 (+2510276us) service_pool.cc:224] Handling call
0709 13:05:12.338728 (+ 79703us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:12.338927 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.339106 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.339142 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.339181 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:12.339218 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.339272 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.339383 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15608576}
I20260709 13:05:12.266291 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.341976 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2380) took 2576 ms. Trace:
I20260709 13:05:12.300091 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.342407 15084 rpcz_store.cc:276] 0709 13:05:09.765592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.328569 (+2562977us) service_pool.cc:224] Handling call
0709 13:05:12.341160 (+ 12591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.341553 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.341595 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.341651 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:12.341710 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.341772 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.341905 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.343078 15839 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:12.343309 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1516) took 2345 ms. Trace:
I20260709 13:05:12.300951 21192 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:12.295053 21308 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.303490 15080 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:12.303622 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:12.344797 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.322829 21042 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.345266 15163 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:12.345757 15080 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:12.346587 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.346753 15163 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:12.347000 21042 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.297056 21269 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:12.347534 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1454) took 2035 ms. Trace:
I20260709 13:05:12.304593 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:11.687996 21303 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.348879 21303 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:12.347937 15157 rpcz_store.cc:276] 0709 13:05:10.312356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.019145 (+706789us) service_pool.cc:224] Handling call
0709 13:05:11.019484 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:11.622433 (+602949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.626665 (+  4232us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:11.664974 (+ 38309us) log.cc:844] Serialized 52 byte log entry
0709 13:05:12.323089 (+658115us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:12.323138 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.323224 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.347337 (+ 24113us) raft_consensus.cc:1685] finished
0709 13:05:12.347354 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.347475 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1545,"raft.queue_time_us":93998,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":12965,"thread_start_us":1128520,"threads_started":2,"wal-append.queue_time_us":675884}
I20260709 13:05:12.304742 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:12.306753 15154 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:12.309321 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:12.343720 15846 rpcz_store.cc:276] 0709 13:05:09.997961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.001885 (+  3924us) service_pool.cc:224] Handling call
0709 13:05:10.027046 (+ 25161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.483854 (+1456808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.343037 (+859183us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400822154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:12.343255 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2935498624}
I20260709 13:05:12.311378 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:12.312323 21450 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:11.741400 21362 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.182734 21416 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.878s	user 0.010s	sys 0.011s
W20260709 13:05:12.315770 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 795369us
W20260709 13:05:12.316461 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 277797us
I20260709 13:05:12.318078 21436 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.320416 21290 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:12.321285 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1508) took 2521 ms (client timeout 2059 ms). Trace:
W20260709 13:05:12.322387 21404 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 13:05:12.323135 15162 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.343676 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.365636 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:12.368182 15079 rpcz_store.cc:269] 0709 13:05:09.799649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.875973 (+ 76324us) service_pool.cc:224] Handling call
0709 13:05:12.321067 (+2445094us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002373d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:12.321213 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2737217408}
I20260709 13:05:12.369508 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:12.347875 21269 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.370834 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.369550 15086 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:11.691797 21383 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.374068 21383 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.375767 15084 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:12.378269 21383 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.372084 15081 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:11.681639 21439 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.382874 15079 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:12.386435 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2381) took 2620 ms. Trace:
I20260709 13:05:12.386765 15079 rpcz_store.cc:276] 0709 13:05:09.765881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.380130 (+2614249us) service_pool.cc:224] Handling call
0709 13:05:12.382706 (+  2576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.386163 (+  3457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.386379 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.390089 14981 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 13 to 14, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 14 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:12.391669 14979 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 357) took 52102 ms (client timeout 15000 ms). Trace:
W20260709 13:05:12.382948 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.385067 21192 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:12.385504 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.392362 14981 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 13 to 14, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 14 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.398420 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:12.400488 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:12.372952 21404 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.371060 21416 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.418164 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:12.407588 14981 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 9 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:12.421506 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.379160 21362 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.406733 21192 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:12.434270 15229 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:12.392139 14979 rpcz_store.cc:269] 0709 13:04:20.288733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.289097 (+   364us) service_pool.cc:224] Handling call
0709 13:05:12.391417 (+52102320us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:05.055800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:05.073527 (+ 17727us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:05.073579 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:05:05.073595 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:05.074200 (+   605us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:05.074298 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:05.074325 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:05:05.074738 (+   413us) tablet.cc:701] Row locks acquired
0709 13:05:05.074793 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:05:05.074955 (+   162us) write_op.cc:270] Start()
0709 13:05:05.075093 (+   138us) write_op.cc:276] Timestamp: P: 1783602305074888 usec, L: 0
0709 13:05:05.075128 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:05:05.218337 (+143209us) log.cc:844] Serialized 1380 byte log entry
0709 13:05:05.838650 (+620313us) op_driver.cc:464] REPLICATION: finished
0709 13:05:05.901470 (+ 62820us) write_op.cc:301] APPLY: starting
0709 13:05:05.901593 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:05.904533 (+  2940us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:05.905733 (+  1200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:05.961411 (+ 55678us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:06.004963 (+ 43552us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:05:06.005026 (+    63us) write_op.cc:312] APPLY: finished
0709 13:05:06.005469 (+   443us) log.cc:844] Serialized 41 byte log entry
0709 13:05:06.005668 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.005756 (+    88us) write_op.cc:454] Released schema lock
0709 13:05:06.005814 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:06.555978 (+550164us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9dfe28 00007f0d4341a581 00007f0d43419d58 00007f0d4341e9f1 00007f0d4341e9a9 00007f0d4341e939 00007f0d4341e901 00007f0d4341db2d 000000000038f354 000000000038f199
0709 13:05:06.556238 (+   260us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.556271 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1111630,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":62537,"apply.run_cpu_time_us":12219,"apply.run_wall_time_us":126527,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16685,"num_ops":3,"prepare.queue_time_us":591368,"prepare.run_cpu_time_us":42834,"prepare.run_wall_time_us":1638648,"raft.queue_time_us":76671,"raft.run_cpu_time_us":13743,"raft.run_wall_time_us":50322,"replication_time_us":763337,"spinlock_wait_cycles":85030144,"thread_start_us":790993,"threads_started":4,"wal-append.queue_time_us":302644,"wal-append.run_cpu_time_us":19448,"wal-append.run_wall_time_us":1491525}]]}
I20260709 13:05:12.439132 21269 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.439697 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.440505 15236 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:12.423012 21383 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.425807 21230 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:12.446053 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45658.3us
W20260709 13:05:12.448904 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.503s	user 0.015s	sys 0.027s
I20260709 13:05:12.412328 21439 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:12.453612 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.451648 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 207684us
W20260709 13:05:12.457799 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 134146us
I20260709 13:05:12.470003 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.470974 15229 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:12.471982 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.470973 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.481739 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.455395 21290 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:12.484951 15234 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:12.490201 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:12.495632 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78574.1us
I20260709 13:05:12.470392 21301 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:12.501157 21194 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:12.501329 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.496217 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.455786 21383 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.510803 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:12.511358 21369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.022s	sys 0.029s
W20260709 13:05:12.511737 21369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.022s	sys 0.029s
I20260709 13:05:12.511618 21362 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.518481 15162 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:12.513720 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.502660 21230 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:12.499423 21301 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:12.507310 15051 connection.cc:256] RPC call timeout handler was delayed by -1.306156177000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 13:05:12.526293 15229 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:12.528267 15231 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:05:12.528203 14981 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:12.528926 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:12.530006 21308 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:12.530423 21308 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.531473 15080 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:05:12.532047 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.493552 21192 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.530486 21281 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:12.530900 21308 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.529773 14981 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 9 to 12, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
W20260709 13:05:12.539043 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85929.9us
I20260709 13:05:12.521579 21269 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:12.549196 21269 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:12.549477 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:12.547928 14981 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 11 to 12, leader changed from <none> to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193), now has a pending config: VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193), VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194), VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195), VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: NON_VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:12.554322 14981 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.544816 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.549660 21269 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:12.557485 15229 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:12.543699 15051 connection.cc:256] RPC call timeout handler was delayed by -1.1882561770000848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 13:05:12.558632 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.543291 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.525617 21230 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.558053 14981 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.531340 21301 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:12.568028 15087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3013) took 2800 ms (client timeout 2392 ms). Trace:
I20260709 13:05:12.525333 21438 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:05:12.581614 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.582423 15232 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:12.572700 15087 rpcz_store.cc:269] 0709 13:05:09.767907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.024196 (+1256289us) service_pool.cc:224] Handling call
0709 13:05:12.543130 (+1518934us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54000dc0d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:12.567935 (+ 24805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466793856}
W20260709 13:05:12.589583 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46145.4us
I20260709 13:05:12.526036 21290 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.567874 21230 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:12.593922 15229 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:12.600374 15088 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:12.559845 21452 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 13:05:12.609321 15088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3025) took 2574 ms (client timeout 3149 ms). Trace:
W20260709 13:05:12.612160 15088 rpcz_store.cc:269] 0709 13:05:10.034624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.502515 (+2467891us) service_pool.cc:224] Handling call
0709 13:05:12.609257 (+106742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.590764 21416 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.606729 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:12.615413 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1502) took 2991 ms. Trace:
I20260709 13:05:12.615231 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.617712 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.620604 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.621403 15087 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:12.613019 21404 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.621842 15230 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:12.624214 15085 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:12.625973 14981 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 11 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 12 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:12.628474 21459 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.819s	user 0.005s	sys 0.026s
I20260709 13:05:12.629981 21466 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:12.630779 21466 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.631139 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.632206 15233 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:05:12.632905 21463 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:05:12.633709 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.634610 21482 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.634951 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:12.635251 21482 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.631139 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.637020 21436 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:12.637434 20916 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:12.638171 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2374) took 2890 ms. Trace:
I20260709 13:05:12.638742 20916 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:12.639322 20916 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:12.638674 15085 rpcz_store.cc:276] 0709 13:05:09.748146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.254647 (+2506501us) service_pool.cc:224] Handling call
0709 13:05:12.623791 (+369144us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:12.623986 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.637831 (+ 13845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.638080 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25405184}
I20260709 13:05:12.640504 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1534) took 1281 ms. Trace:
I20260709 13:05:12.641353 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.641942 21416 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:12.642166 15085 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:12.642478 21416 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:12.616147 15083 rpcz_store.cc:276] 0709 13:05:09.624444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.645309 (+2020865us) service_pool.cc:224] Handling call
0709 13:05:12.355563 (+710254us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfeee 0000000000375939 00007f0d449331e0 00007f0d449fc0c4 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:12.355736 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.355933 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:12.462083 (+106150us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.615012 (+152929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.615081 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:12.615132 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.615222 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.615362 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153737,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"spinlock_wait_cycles":4551040,"thread_start_us":219595,"threads_started":3}
I20260709 13:05:12.628227 14981 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 16 to 17, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 17 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.635473 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:12.647341 15084 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:12.615190 21304 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:12.656409 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72603.5us
I20260709 13:05:12.595783 21194 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:05:12.663882 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.640975 15841 rpcz_store.cc:276] 0709 13:05:11.358768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.492073 (+133305us) service_pool.cc:224] Handling call
0709 13:05:11.492521 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.493276 (+   755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.493319 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.494018 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.641843 (+147825us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:11.641885 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.641967 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.627747 (+985780us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:12.640172 (+ 12425us) raft_consensus.cc:1685] finished
0709 13:05:12.640197 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.640423 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5747,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":161,"spinlock_wait_cycles":218205056,"thread_start_us":147313,"threads_started":1,"wal-append.queue_time_us":315174}
I20260709 13:05:12.666754 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:12.663882 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.637470 21436 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.614053 20976 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:05:12.643496 21450 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.650923 21400 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:12.664203 14981 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:12.677209 21464 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:05:12.640383 20916 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.678351 21304 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:12.683327 21436 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.687613 21466 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.696621 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.697970 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.698415 21438 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:12.693316 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 118450us
W20260709 13:05:12.692252 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92677.4us
I20260709 13:05:12.708724 15230 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:12.688374 21439 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.701124 21042 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:12.693882 21266 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:12.789451 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 95263.4us
W20260709 13:05:12.798741 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.705986 21281 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.804777 15088 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
W20260709 13:05:12.793186 21472 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:05:12.709436 21383 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.705253 21400 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:12.826165 21479 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:05:12.827222 21474 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:05:12.827445 21303 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:12.829064 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.829661 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.829908 21469 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:05:12.833312 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:12.832823 21080 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:12.834095 21394 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:12.833391 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 120659us
I20260709 13:05:12.835296 21452 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:12.839110 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.828948 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:12.841419 15237 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:12.841692 21304 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.842576 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.844343 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.845089 14981 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 8 to 14, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 14 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.849157 21489 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:12.849977 21452 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:12.852505 21439 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:12.910885 20981 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:12.888653 21467 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.315s	user 0.001s	sys 0.001s
I20260709 13:05:12.910998 14981 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 4 to 13, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:12.853462 21362 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:12.854516 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3020) took 2951 ms. Trace:
I20260709 13:05:12.917196 15088 rpcz_store.cc:276] 0709 13:05:09.886202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.690784 (+2804582us) service_pool.cc:224] Handling call
0709 13:05:12.793867 (+103083us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:12.837342 (+ 43475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.837670 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:12.857002 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 171977us
W20260709 13:05:12.858186 21470 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:05:12.848470 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 52787.3us
I20260709 13:05:12.869203 20916 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:12.824329 21464 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:05:12.877235 21290 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.878816 21266 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.821816 21042 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:12.881794 21281 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:12.862890 21383 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.889884 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.932698 15236 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:12.894255 21303 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.903632 15237 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:12.902930 15232 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:05:12.839689 21482 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.923579 14981 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 10 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.941013 15232 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:12.907402 21308 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.950739 15127 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:12.891031 21301 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.929615 15841 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:12.930330 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.939162 15228 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:12.931870 21042 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.943429 15840 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:12.943751 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.944038 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.944412 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:12.944672 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.945416 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.946125 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.946540 21281 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:12.947114 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.954208 15843 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:12.947201 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.947624 21383 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:12.949247 15236 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:05:12.923154 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 89908.9us
I20260709 13:05:12.964279 15847 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:12.962839 15846 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:12.958403 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1507) took 3159 ms. Trace:
I20260709 13:05:12.958424 15088 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:12.964771 15079 rpcz_store.cc:276] 0709 13:05:09.799311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.387573 (+2588262us) service_pool.cc:224] Handling call
0709 13:05:12.387931 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.388189 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:12.495835 (+107646us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.957897 (+462062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.957952 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:12.958011 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.958101 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.958335 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11407,"prepare.queue_time_us":156088,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":6253,"raft.queue_time_us":107247,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":85215,"spinlock_wait_cycles":55078912,"thread_start_us":509488,"threads_started":3,"wal-append.queue_time_us":125159}
I20260709 13:05:12.970109 15237 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:05:12.969063 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 118912us
I20260709 13:05:12.974855 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:12.974752 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:12.979176 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.980829 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:12.981208 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.982605 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:12.983341 21383 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:12.985039 21080 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:12.985473 21080 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.985823 21080 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:12.956532 21489 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:12.980449 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.996003 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:12.989576 15236 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:12.983400 15088 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:12.990486 15233 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:12.995970 15237 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:05:12.998028 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:12.999017 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:12.997613 15237 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 9
I20260709 13:05:13.014981 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:13.015267 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 112825us
W20260709 13:05:13.016518 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 180024us
I20260709 13:05:13.016620 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.018438 15079 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.018515 15839 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:13.006433 15237 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:05:12.998853 21400 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:13.021631 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:13.020927 15237 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.023310 14981 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 7 to 10, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:13.028530 15235 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:13.028753 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.036734 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.031406 14981 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.044919 15160 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:13.048621 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.004096 21489 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:13.050979 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55067.3us
I20260709 13:05:13.060388 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1535) took 1688 ms. Trace:
I20260709 13:05:13.033926 20916 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.062979 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.064049 15082 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:13.053437 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:13.060798 15842 rpcz_store.cc:276] 0709 13:05:11.359265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.475237 (+115972us) service_pool.cc:224] Handling call
0709 13:05:13.036581 (+1561344us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b58001ffe98. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:05:13.047977 (+ 11396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3644160}
I20260709 13:05:13.070258 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.047940 14981 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 9 to 13, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 13 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.069772 20916 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:13.075131 20916 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:13.075357 21490 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:13.076012 21490 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.076442 21490 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.076388 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.077549 15156 raft_consensus.cc:2410] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:13.076612 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.075501 20916 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.081315 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.086731 20916 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:13.087399 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.088327 15162 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:13.090598 20916 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.071280 21494 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:13.093395 21494 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:13.026423 21269 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.095741 21269 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.098163 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.099920 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.100059 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.101392 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:13.102147 15159 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:13.102903 21192 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:13.103322 21192 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:13.103772 21192 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.103703 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.103339 15158 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:13.105651 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.106205 15157 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:13.107750 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.093860 21494 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.108640 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:13.109233 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.109447 21400 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:13.109843 15157 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:13.110411 21400 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:13.108664 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.110936 21400 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:13.111361 21400 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.112460 21362 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:13.112401 21301 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.113124 21362 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:13.114470 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.114233 21362 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.115494 21362 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.115855 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.116655 15159 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:13.117074 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68396.8us
I20260709 13:05:13.123948 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.105141 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.037112 21450 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:13.127601 21450 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:13.128226 21450 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:13.129151 15158 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:13.131117 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:13.131690 15839 raft_consensus.cc:2410] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:13.131042 21489 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.132539 21303 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:13.134217 21303 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:13.135159 21301 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:13.135499 21301 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:13.136289 21301 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.139634 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.119832 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.143178 15842 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:13.144279 21303 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:13.150522 14981 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:13.153069 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 202470us
I20260709 13:05:13.154389 21192 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.154980 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.141775 15237 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:05:13.142205 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2373) took 3326 ms. Trace:
I20260709 13:05:13.157796 21400 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.158100 15082 rpcz_store.cc:276] 0709 13:05:09.747822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.660652 (+1912830us) service_pool.cc:224] Handling call
0709 13:05:13.063567 (+1402915us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:13.063828 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.074179 (+ 10351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.074440 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54532480}
I20260709 13:05:13.159114 21502 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:13.159809 21502 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.160537 21502 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.161736 21450 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.162662 21502 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.163303 21042 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:13.164651 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.165087 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.165342 15158 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:13.165812 15155 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 11: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:13.165740 21281 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:13.166513 21281 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.166971 21281 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.167857 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.168534 15158 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:13.169057 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.169278 21383 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.169685 15159 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:13.170302 21192 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.160029 21400 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.169955 21383 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.170969 15154 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:13.119522 21080 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.175966 21383 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.176620 21080 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.168505 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.177006 21436 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:13.180346 21436 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.092181 21230 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.181710 15848 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:13.184584 21230 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.159715 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.209187 15082 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:13.152753 14981 catalog_manager.cc:5697] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 2 to 11. New cstate: current_term: 11 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:13.198310 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 111030us
I20260709 13:05:13.142486 21301 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:13.219286 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62769.3us
I20260709 13:05:13.219985 15810 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.214766 15079 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:13.221763 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.230965 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.236479 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.245816 21080 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:13.246309 21080 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.246670 21080 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.247900 21490 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.245405 21494 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.249933 21383 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:13.251354 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.252609 15846 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:13.251924 15082 raft_consensus.cc:2376] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17. Re-sending same reply.
I20260709 13:05:13.254892 15079 raft_consensus.cc:2368] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:13.243094 21394 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:13.256153 21394 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:13.257506 21394 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:13.258442 21394 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:13.261225 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.262493 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.263375 15084 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:13.264007 21230 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:13.264214 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.264468 21230 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.265352 21230 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.266535 21230 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.267256 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.268121 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.268481 21400 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.274431 21505 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:13.276584 15127 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.253737 21495 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:05:13.279366 21495 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:13.279757 21495 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.282532 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.283438 15088 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.267119 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.285844 15081 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:13.286199 15080 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:13.286820 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1509) took 3297 ms. Trace:
I20260709 13:05:13.287330 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3019) took 3402 ms. Trace:
I20260709 13:05:13.287698 15081 rpcz_store.cc:276] 0709 13:05:09.884766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.405937 (+2521171us) service_pool.cc:224] Handling call
0709 13:05:13.285454 (+879517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.286285 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.287158 (+   873us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000992d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:13.287280 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953585536}
I20260709 13:05:13.288435 15085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:56652
I20260709 13:05:13.289182 15085 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:13.289528 15085 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.294164 21080 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.296499 21080 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.297685 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2586) took 3622 ms. Trace:
W20260709 13:05:13.300398 21481 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.666s	user 0.011s	sys 0.000s
I20260709 13:05:13.304484 20870 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:13.305644 15082 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.337248 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1456) took 1664 ms. Trace:
I20260709 13:05:13.337667 15160 rpcz_store.cc:276] 0709 13:05:11.673831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.874632 (+1200801us) service_pool.cc:224] Handling call
0709 13:05:12.874973 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.050053 (+175080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.337015 (+286962us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540033ed54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:13.337183 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390175104}
W20260709 13:05:13.305964 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61057.9us
I20260709 13:05:13.306283 21383 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:13.306499 21266 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:13.307029 21042 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:13.346746 15085 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.308019 21308 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.308775 21194 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:13.308947 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.313900 21488 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:13.316998 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3014) took 3548 ms. Trace:
I20260709 13:05:13.369206 15086 rpcz_store.cc:276] 0709 13:05:09.768177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.380228 (+2612051us) service_pool.cc:224] Handling call
0709 13:05:12.421616 (+ 41388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.443708 (+ 22092us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:12.533519 (+ 89811us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.678075 (+144556us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.680655 (+  2580us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.696555 (+ 15900us) log.cc:844] Serialized 26 byte log entry
0709 13:05:12.696721 (+   166us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:12.696776 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.696897 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.256955 (+560058us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b180042ca60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:13.302676 (+ 45721us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b180042ca60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:13.316726 (+ 14050us) raft_consensus.cc:1685] finished
0709 13:05:13.316751 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.316926 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86870,"prepare.queue_time_us":157467,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1906,"raft.queue_time_us":359424,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":20827,"spinlock_wait_cycles":22995072,"thread_start_us":73068,"threads_started":2,"wal-append.queue_time_us":63218}
I20260709 13:05:13.372350 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.322949 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.323735 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.287120 15080 rpcz_store.cc:276] 0709 13:05:09.989772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.285632 (+3295860us) service_pool.cc:224] Handling call
0709 13:05:13.286018 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.286597 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.286766 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.327235 20981 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:13.327696 21281 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.250909 21494 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.280630 21495 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.328971 21416 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.332631 21362 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.413673 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.333449 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:13.333755 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3021) took 3447 ms. Trace:
W20260709 13:05:13.415984 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 93103.5us
I20260709 13:05:13.333956 21400 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.305505 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.420239 15081 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:13.420395 15087 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.336820 15231 rpcz_store.cc:276] 0709 13:05:09.674934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.759726 (+ 84792us) service_pool.cc:224] Handling call
0709 13:05:09.759994 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.531948 (+2771954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.297355 (+765407us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b5400319054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:13.297618 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6332601216}
I20260709 13:05:13.377030 21488 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.377485 20981 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:13.365134 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 340448us
I20260709 13:05:13.254029 21499 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:05:13.434264 20981 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.435189 21042 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.435345 20981 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.437288 21400 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.439721 21400 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.439647 21394 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:13.440881 21394 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:13.441416 21230 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:13.441833 21230 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.442121 21495 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.442613 21230 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.443845 21230 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:13.445171 21416 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.439685 21452 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:05:13.446503 21452 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:13.446823 21452 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.447585 21452 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.374598 15086 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.376811 15155 raft_consensus.cc:2410] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:13.337255 20870 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.396936 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.450075 15079 raft_consensus.cc:2376] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14. Re-sending same reply.
I20260709 13:05:13.406100 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.351764 21266 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:13.412030 15085 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.364449 21438 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:05:13.417198 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 165969us
W20260709 13:05:13.453718 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 88854.4us
W20260709 13:05:13.451543 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 142696us
I20260709 13:05:13.416298 15088 rpcz_store.cc:276] 0709 13:05:09.886537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.284050 (+3397513us) service_pool.cc:224] Handling call
0709 13:05:13.284423 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.284614 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.284643 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.284687 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:13.284721 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.284778 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.333702 (+ 48924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.365110 21194 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.339852 15083 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:13.356777 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.419150 21497 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.424250 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:13.424168 21395 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:13.424652 21192 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:05:13.424062 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90692.9us
I20260709 13:05:13.465072 15084 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:13.431073 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1512) took 3393 ms. Trace:
I20260709 13:05:13.465531 15081 rpcz_store.cc:276] 0709 13:05:10.037247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.288559 (+3251312us) service_pool.cc:224] Handling call
0709 13:05:13.288957 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.420654 (+131697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.430828 (+ 10174us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540014df54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:13.431005 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301438848}
I20260709 13:05:13.432801 21488 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.459506 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:13.463383 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 106870us
I20260709 13:05:13.465984 15080 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:13.470891 15088 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:13.471706 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.473395 15083 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:13.473161 20870 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.474354 21490 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.474660 20870 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.474872 15083 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:05:13.475283 15083 pending_rounds.cc:85] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 11
I20260709 13:05:13.475487 15083 pending_rounds.cc:107] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:05:13.475733 15083 raft_consensus.cc:2889] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.476141 15084 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:13.476823 21510 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:13.477859 21042 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.462191 21395 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.479835 21395 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.480881 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.481156 21395 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.481833 21487 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:13.482389 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.461851 15087 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.467317 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56652 (request call id 1511) took 3457 ms. Trace:
I20260709 13:05:13.482503 21487 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.483664 15085 rpcz_store.cc:276] 0709 13:05:10.000426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.288233 (+3287807us) service_pool.cc:224] Handling call
0709 13:05:13.451205 (+162972us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":200}
I20260709 13:05:13.485013 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.460377 21383 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:13.485805 21450 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:13.486328 21488 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:13.485822 21383 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.486848 21383 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.469882 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.483469 21266 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:13.485374 21281 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.450438 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.489678 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:13.420167 21362 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.453949 21438 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.495976 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1513) took 3414 ms. Trace:
I20260709 13:05:13.496474 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.496364 15080 rpcz_store.cc:276] 0709 13:05:10.081621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.396363 (+3314742us) service_pool.cc:224] Handling call
0709 13:05:13.458009 (+ 61646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.470466 (+ 12457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.495786 (+ 25320us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b54001b03d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:13.495921 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76948736}
W20260709 13:05:13.497571 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47207us
I20260709 13:05:13.498138 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:13.493285 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.492548 21487 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:13.495215 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.499533 15162 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:13.496078 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.509456 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:13.509239 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.501004 15157 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:05:13.500367 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:13.515863 15160 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:13.467579 21304 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.515321 15154 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:13.516718 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:13.491575 21450 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:13.500469 21487 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.516076 21192 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:13.539073 21394 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.539675 21394 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.541941 21394 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:13.530359 21510 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:05:13.546200 21510 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:13.546577 21510 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.547554 21510 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.530500 21489 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:13.549263 21489 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.550335 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:13.550266 21489 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:13.551364 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.520367 21303 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:13.516637 21509 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:13.529639 21281 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.556165 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.500816 21301 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:05:13.556825 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:13.557631 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:13.560868 15230 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:13.537776 21487 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.566016 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:13.552606 21490 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:13.564553 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:13.564949 21362 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.537405 21192 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.562484 15228 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:13.555872 21304 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:13.566766 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.571628 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:13.566771 15232 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.571703 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.548799 21510 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.553591 21303 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:13.572783 15231 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:13.579933 15229 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:13.561954 21266 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:13.596441 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 96836.4us
I20260709 13:05:13.601955 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.603359 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.604117 15839 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.597340 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:13.601037 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.613672 15841 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.601389 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.615393 15235 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:05:13.615531 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:13.617616 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.617424 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.621300 15843 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.626046 21516 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:13.626963 20916 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:13.627502 20916 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:13.627733 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.628613 20916 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.629185 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.580410 21192 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.601809 21303 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:13.630316 20916 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.621310 21362 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:13.562258 21281 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:13.648440 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 139065us
I20260709 13:05:13.644636 15161 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:13.663774 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.528535 21194 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:13.659937 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 166726us
I20260709 13:05:13.661244 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.667376 15085 raft_consensus.cc:2376] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18. Re-sending same reply.
I20260709 13:05:13.632089 21304 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:13.661652 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.631163 21505 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.664769 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:13.555068 21438 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.667096 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.675441 15232 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:13.677075 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.677109 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:13.680954 15229 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:05:13.683373 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.685804 15128 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:13.686967 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:13.689915 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 139650us
I20260709 13:05:13.690809 15201 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.697440 20870 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:13.699124 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.702102 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.661007 21362 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:13.702334 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.663275 21518 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:13.667411 21494 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.701833 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.705443 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.707271 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.709656 15084 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.711615 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.677975 21526 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.719329 15840 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:13.721088 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:13.723667 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1519) took 1210 ms. Trace:
I20260709 13:05:13.724391 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2387) took 2211 ms. Trace:
I20260709 13:05:13.724159 15086 rpcz_store.cc:276] 0709 13:05:12.513709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.451936 (+938227us) service_pool.cc:224] Handling call
0709 13:05:13.452223 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.462766 (+ 10543us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:13.659469 (+196703us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.723251 (+ 63782us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.723307 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:13.723358 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.723433 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.723593 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146561,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":92,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":202060,"threads_started":2}
I20260709 13:05:13.725049 15082 rpcz_store.cc:276] 0709 13:05:11.512589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.406254 (+1893665us) service_pool.cc:224] Handling call
0709 13:05:13.406604 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.414609 (+  8005us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:13.678225 (+263616us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.724014 (+ 45789us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.724084 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:13.724139 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.724231 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.724336 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193313,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":82,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":222190,"threads_started":2}
I20260709 13:05:13.701872 21194 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.733210 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:13.734549 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:13.741815 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 51115.5us
I20260709 13:05:13.744261 15082 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:13.618048 21450 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.700893 21395 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:05:13.716254 21362 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.747373 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.710129 20870 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:13.747601 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45461.2us
I20260709 13:05:13.759254 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.759903 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.764562 15847 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.767268 15840 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:13.779281 15845 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:13.772214 15082 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:05:13.772457 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:13.747380 21531 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:13.764551 21438 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.738003 21494 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.769145 20870 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.793951 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.734985 21518 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.771430 21526 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:13.804363 15840 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.802539 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.780784 21304 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.819469 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.822080 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.831415 21498 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:13.832466 21498 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:13.833767 15230 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:13.834321 15845 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:13.772425 21395 consensus_queue.cc:1243] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:05:13.845587 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.849193 21281 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:13.781096 21516 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 13:05:13.850236 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.850917 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.857167 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.858458 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.859051 15845 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:13.861703 15085 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:13.862847 15848 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.863245 15841 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:13.859440 15845 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:13.812429 21383 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:13.874279 15847 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:13.882835 21518 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:13.885064 21505 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.886577 15845 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:13.887719 15845 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.891897 15840 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:13.893949 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:13.894719 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.878786 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:13.822540 21526 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:13.897534 15841 raft_consensus.cc:2410] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:05:13.897866 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:13.915998 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 229092us
I20260709 13:05:13.922729 15845 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:13.879571 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:13.931764 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:13.932667 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:13.946409 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:13.840634 21498 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:13.967090 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 145072us
I20260709 13:05:13.967941 15839 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:13.952106 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:13.952306 21400 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:13.971032 15842 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:13.819617 21301 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
W20260709 13:05:13.953229 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73802.5us
I20260709 13:05:13.953795 21544 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:13.957892 21494 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:13.799578 21524 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 13:05:13.959132 21490 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:13.961210 21452 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.944022 21526 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.951645 15163 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:13.970453 15201 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:13.952526 21545 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:05:13.975842 15080 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:13.980064 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 315335us
I20260709 13:05:13.983283 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:13.983069 21549 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:13.972225 21269 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:13.898075 21543 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:13.975363 21281 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:13.998499 15839 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:14.004909 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:13.992259 21488 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.002s
I20260709 13:05:14.017640 15081 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:13.987726 21490 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.987181 21524 consensus_queue.cc:1243] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:14.016551 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:14.022406 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.018617 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:14.020576 15848 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:13.982052 21400 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:14.009946 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:14.021201 15162 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:13.989472 21439 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.026592 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.021807 21490 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:14.026646 21281 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:14.044799 21400 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.008476 21229 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:14.021845 21304 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:05:14.026121 21498 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.027395 21301 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:05:14.048420 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:14.053462 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.052116 21042 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.004s
I20260709 13:05:14.054800 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.051187 15158 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:14.054225 15839 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:14.058687 21383 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:14.052511 21439 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:14.026870 21269 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:14.059705 21509 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:14.060142 21544 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:14.050276 15843 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:14.065158 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1525) took 1074 ms. Trace:
W20260709 13:05:14.065297 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.048269 21524 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:14.065619 21303 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.068467 15160 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:14.115638 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:14.123291 15203 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:14.056787 21536 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:14.115350 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:14.068130 21042 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:14.130918 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:14.134902 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:14.118646 15088 rpcz_store.cc:276] 0709 13:05:12.990752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.472028 (+481276us) service_pool.cc:224] Handling call
0709 13:05:13.472391 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:13.489356 (+ 16965us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:13.621227 (+131871us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.816065 (+194838us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:13.848939 (+ 32874us) log.cc:844] Serialized 52 byte log entry
0709 13:05:13.849138 (+   199us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:13.849184 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.849268 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.064917 (+215649us) raft_consensus.cc:1685] finished
0709 13:05:14.064941 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.065094 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124685,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":2368,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":152,"thread_start_us":247241,"threads_started":2,"wal-append.queue_time_us":203156}
I20260709 13:05:14.116751 21509 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:14.118440 21544 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:14.145397 15159 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:14.150506 15847 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:05:14.155714 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.138108 21531 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:14.129097 21536 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:14.157505 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.160054 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:14.161990 21439 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:14.165215 21269 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.166435 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:14.167572 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:14.138823 21303 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:14.169866 15156 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:14.173455 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:14.179656 15839 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:05:14.181393 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.181790 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:14.182000 15159 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:14.185138 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:14.145363 21524 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:14.187723 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.147348 21552 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:14.192027 15088 pending_rounds.cc:85] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 8
I20260709 13:05:14.192399 15088 pending_rounds.cc:107] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:14.192657 15088 raft_consensus.cc:2889] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.192912 15160 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:14.160772 21536 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:14.201272 15081 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:14.203172 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:14.183691 21301 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:14.207441 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:14.188748 21553 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.139190 21281 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.190187 21229 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:14.209657 21301 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.217264 21301 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:14.218845 21535 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:05:14.219640 21524 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.216698 20870 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:14.223445 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.224467 15839 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:14.228322 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.229133 15160 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:14.207281 21303 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:14.238809 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:14.223240 21560 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:05:14.260231 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.263259 15839 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:14.274591 15839 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:14.274318 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.275530 15841 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:05:14.277580 15841 pending_rounds.cc:85] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 12
I20260709 13:05:14.241225 21303 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.278173 15841 pending_rounds.cc:107] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:05:14.256479 21552 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:14.280058 15159 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:14.282899 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79852.8us
I20260709 13:05:14.285863 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:14.289412 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 104202us
I20260709 13:05:14.210474 21509 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.299124 15841 raft_consensus.cc:2889] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.300268 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:14.237112 21553 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.300750 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.312872 15840 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:14.288460 20981 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:14.323413 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.324018 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.291390 21192 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.632s	user 0.006s	sys 0.005s
I20260709 13:05:14.315474 20870 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:14.317050 21400 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:14.287730 21552 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:14.225744 21544 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.293741 21303 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:14.335959 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.336501 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.339376 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.312356 21509 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:14.292831 21229 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:05:14.336784 15160 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
W20260709 13:05:14.340405 21487 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.744s	user 0.012s	sys 0.000s
I20260709 13:05:14.333912 21552 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:14.357153 21561 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:05:14.364261 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.331488 21192 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:14.369520 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.370939 15841 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [14.13-16.14]
I20260709 13:05:14.371516 15159 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:14.375207 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.358268 21400 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:14.376540 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79222.9us
W20260709 13:05:14.376791 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 76734.4us
I20260709 13:05:14.377763 15841 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:05:14.378927 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:14.338613 21567 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:05:14.382719 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:14.384920 15081 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:14.386767 21192 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:14.361690 21487 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:14.389772 21487 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:14.391119 21301 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:14.393307 21301 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:14.393929 15846 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:14.396943 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:14.447806 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.512920 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.513873 15162 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:14.510591 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:14.521502 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:14.521870 15081 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:14.522141 15088 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:14.367161 21490 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.543136 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.544564 15162 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:14.515811 21192 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:14.548705 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.548885 21564 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:05:14.551772 21563 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:05:14.562697 20981 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:14.566602 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:14.568118 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 168127us
W20260709 13:05:14.569555 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.570331 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:14.570616 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.605505 15232 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:05:14.613296 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.608074 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1483) took 1033 ms. Trace:
I20260709 13:05:14.619149 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.567741 21192 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:14.619149 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:14.627244 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.623586 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.625365 15237 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:14.628080 15159 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:14.619920 15154 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:14.628402 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 605730us
I20260709 13:05:14.627576 15228 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:14.629133 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:14.629194 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:14.627928 15813 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.640s (SENT)
I20260709 13:05:14.622925 21573 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:14.630007 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:14.616313 15155 rpcz_store.cc:276] 0709 13:05:13.574692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.652693 (+ 78001us) service_pool.cc:224] Handling call
0709 13:05:13.679263 (+ 26570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.679483 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:13.705413 (+ 25930us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.980500 (+275087us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.103158 (+122658us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b18003d1140. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1
0709 13:05:14.121515 (+ 18357us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18003d1140. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:14.607613 (+486098us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18003d1140. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:05:14.607756 (+   143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.607804 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:14.607851 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.607916 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.608020 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32503,"prepare.queue_time_us":29103,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":27946,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4897,"spinlock_wait_cycles":30592768,"thread_start_us":245358,"threads_started":1,"wal-append.queue_time_us":348921}
I20260709 13:05:14.575654 20981 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:14.626464 21452 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.003s
W20260709 13:05:14.635877 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 112557us
W20260709 13:05:14.640693 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.617642 21281 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:14.632182 21192 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:14.645901 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.647064 21553 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:14.641326 21531 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:14.643669 15844 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:14.660218 15081 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
W20260709 13:05:14.662668 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.664403 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.665967 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:14.680517 15086 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:14.662082 20870 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:14.657292 21281 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:14.692378 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63319us
W20260709 13:05:14.695171 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.678346 21505 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:14.684899 21524 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:14.661912 21553 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 13:05:14.696769 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 122904us
W20260709 13:05:14.697283 21362 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.900s	user 0.013s	sys 0.000s
I20260709 13:05:14.711877 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.717027 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:14.718153 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:14.720738 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:14.731081 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:14.736616 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1484) took 1161 ms. Trace:
I20260709 13:05:14.737223 15161 rpcz_store.cc:276] 0709 13:05:13.574913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.678367 (+103454us) service_pool.cc:224] Handling call
0709 13:05:13.690838 (+ 12471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.949933 (+259095us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:13.965456 (+ 15523us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.634844 (+669388us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.635293 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.635440 (+   147us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:14.635476 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.635539 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.736350 (+100811us) raft_consensus.cc:1685] finished
0709 13:05:14.736379 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.736542 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":601240,"prepare.queue_time_us":2603,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":4071,"raft.queue_time_us":667668,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":7907,"thread_start_us":267278,"threads_started":2,"wal-append.queue_time_us":121282}
I20260709 13:05:14.742058 21490 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:14.739037 15162 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:14.743471 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:14.745622 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 124160us
I20260709 13:05:14.700524 21505 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.763111 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:14.771519 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 143618us
I20260709 13:05:14.772773 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2401) took 1125 ms. Trace:
I20260709 13:05:14.773259 15080 rpcz_store.cc:276] 0709 13:05:13.647885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.648048 (+   163us) service_pool.cc:224] Handling call
0709 13:05:14.772554 (+1124506us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540014d554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:14.772708 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716062976}
I20260709 13:05:14.771734 15202 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:14.716624 21450 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:05:14.716555 21362 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:14.776368 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 110516us
I20260709 13:05:14.782712 21304 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:14.783152 21304 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:14.784024 21304 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.761085 20916 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:14.786285 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1533) took 1103 ms. Trace:
I20260709 13:05:14.764799 21516 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:14.787190 15839 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:14.761170 21553 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:14.794476 15848 rpcz_store.cc:276] 0709 13:05:13.683001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.683123 (+   122us) service_pool.cc:224] Handling call
0709 13:05:14.786123 (+1103000us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54006cdc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:14.786227 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411267456}
W20260709 13:05:14.803289 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 104266us
I20260709 13:05:14.810966 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:14.780328 21498 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:14.810613 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:14.799340 21565 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:05:14.817710 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.831046 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:14.831198 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:14.838193 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:14.839210 15230 raft_consensus.cc:2368] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16.
I20260709 13:05:14.845862 15088 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:14.848510 21362 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:14.848899 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:14.853226 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.853014 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:14.853792 15230 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:14.799741 21539 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:14.800181 21588 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.817975 21516 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:14.863025 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.832290 21304 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:14.866008 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:14.866958 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 107619us
W20260709 13:05:14.879559 15128 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.755s (SENT)
W20260709 13:05:14.878666 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 88898.5us
I20260709 13:05:14.897643 15232 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:05:14.799527 21577 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.166s
W20260709 13:05:14.855408 21547 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.711s	user 0.014s	sys 0.018s
W20260709 13:05:14.858281 21536 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.649s	user 0.003s	sys 0.010s
W20260709 13:05:14.901803 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.902197 15230 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:14.904583 15232 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:14.775529 21281 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.908829 15228 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:14.911185 15236 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:14.857959 21584 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:14.914956 15237 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:14.830715 21585 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:05:14.918401 15848 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:14.857861 21362 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:05:14.920799 15232 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:14.921602 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:14.927063 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:14.928129 15840 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:05:14.929001 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 165947us
W20260709 13:05:14.952881 15128 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.794s (SENT)
I20260709 13:05:14.953606 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:14.955791 21590 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:14.956504 21303 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.001s
I20260709 13:05:14.957041 21303 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:14.957401 21505 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:14.957755 21557 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:14.907958 21536 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.960006 15233 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:14.959311 21303 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:14.961216 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:14.962286 15232 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:05:14.956507 21590 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:14.966401 21194 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:14.960234 21536 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:14.974123 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:15.012108 15054 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.863s	user 0.022s	sys 0.021s
I20260709 13:05:15.014407 15088 raft_consensus.cc:2368] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19.
I20260709 13:05:14.969362 21590 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:15.024245 21583 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:05:15.024041 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:15.043008 15813 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.059s (SENT)
I20260709 13:05:15.043689 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:15.045058 21516 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.045789 21590 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:15.051530 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1532) took 1369 ms. Trace:
I20260709 13:05:15.052220 15843 rpcz_store.cc:276] 0709 13:05:13.682625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.689667 (+  7042us) service_pool.cc:224] Handling call
0709 13:05:15.051244 (+1361577us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54004a7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:15.051422 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764074880}
W20260709 13:05:15.055994 21526 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.052s	user 0.000s	sys 0.011s
I20260709 13:05:14.924264 21584 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:15.061759 21584 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:15.013105 21487 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:15.068382 21487 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.071182 21487 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:15.072506 21509 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.003s
I20260709 13:05:14.893790 21588 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.074712 21588 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:15.076690 15085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2405) took 1396 ms (client timeout 1794 ms). Trace:
W20260709 13:05:15.077150 15085 rpcz_store.cc:269] 0709 13:05:13.680827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.685322 (+  4495us) service_pool.cc:224] Handling call
0709 13:05:15.076491 (+1391169us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002378d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:15.076631 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330524416}
W20260709 13:05:15.080628 15813 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.095s (SENT)
I20260709 13:05:15.081300 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:15.086329 15054 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.937s	user 0.022s	sys 0.021s
W20260709 13:05:15.087055 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.93778e+06us
W20260709 13:05:15.088078 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 174406us
I20260709 13:05:15.090549 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:15.091259 15842 raft_consensus.cc:2376] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16. Re-sending same reply.
W20260709 13:05:14.969609 21578 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:05:15.100730 15845 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:05:15.087708 15054 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.536s (SENT) [suppressed 30 similar messages]
I20260709 13:05:15.103358 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.105507 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.106611 15228 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:14.932111 21592 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.014055 21580 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:15.111274 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 180121us
W20260709 13:05:15.108930 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 156158us
I20260709 13:05:15.113564 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.114234 15840 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:15.114418 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.115010 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:15.119725 15235 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:15.118472 15232 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:15.120674 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.124038 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.124658 15237 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:15.124985 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44427.6us
I20260709 13:05:15.094944 21439 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:15.128417 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:15.121254 15232 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:15.118886 21605 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:15.131843 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.133862 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:15.145280 15160 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:15.113540 20976 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.134409 21450 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:15.134454 21546 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:15.124532 21505 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.146171 15845 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:15.146406 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:15.147948 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 315339us
I20260709 13:05:15.144119 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:14.972873 21539 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.170s
I20260709 13:05:15.156690 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:15.157361 15159 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.153131 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:15.056582 21526 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.159287 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.159439 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:15.160187 15230 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:15.160792 15843 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:15.155476 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:15.165091 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.165311 15842 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:15.166426 15155 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:15.166267 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:15.159973 15087 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:15.152447 15054 connection.cc:256] RPC call timeout handler was delayed by -1.9298038089999636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
W20260709 13:05:15.168943 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.168792 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.169920 21509 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:15.170450 21604 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:15.153247 21588 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:15.142304 21580 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:15.197695 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:15.172096 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:15.198005 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.198243 15237 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:15.207289 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:15.200857 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.189010 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.202792 15086 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:05:15.171921 15088 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:15.126292 21557 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 13:05:15.174000 21552 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.804s	user 0.007s	sys 0.004s
I20260709 13:05:15.217587 15237 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:15.164759 21553 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:15.219303 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:15.219118 15054 connection.cc:256] RPC call timeout handler was delayed by -1.8335038090000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
I20260709 13:05:15.119541 21592 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.223970 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:15.222922 15054 connection.cc:256] RPC call timeout handler was delayed by -1.8137038090000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 13:05:15.197860 21585 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.225562 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:15.222226 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.228811 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:15.226017 15054 connection.cc:256] RPC call timeout handler was delayed by -1.8101038089999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 13:05:15.197695 21303 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:15.150740 21498 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:05:15.203297 21604 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.229207 15237 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:15.283224 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54478us
I20260709 13:05:15.232534 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1410) took 1617 ms. Trace:
I20260709 13:05:15.286231 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:15.230717 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.158114 21546 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.160636 21505 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:15.288976 15232 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:15.285511 15231 rpcz_store.cc:276] 0709 13:05:13.615416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.616640 (+  1224us) service_pool.cc:224] Handling call
0709 13:05:13.644398 (+ 27758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.551393 (+906995us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b180033dd80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9dcbfb 00007f0d3f9e764b
0709 13:05:14.551549 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.551571 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.552116 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.677409 (+125293us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:14.677445 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.677519 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.223635 (+546116us) raft_consensus.cc:1685] finished
0709 13:05:15.223662 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.232456 (+  8794us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5986,"raft.queue_time_us":74505,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":11107,"spinlock_wait_cycles":6695168,"thread_start_us":221266,"threads_started":2,"wal-append.queue_time_us":226287}
W20260709 13:05:15.221222 21571 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:15.326366 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:15.326889 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:15.282140 20981 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.002s
I20260709 13:05:15.155412 20976 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:15.345014 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 143636us
I20260709 13:05:15.346063 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:15.282612 21487 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:15.351593 15160 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:15.293354 21490 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:15.361521 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:15.381731 15232 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.367224 15237 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
W20260709 13:05:15.499902 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.287999 21604 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:15.550513 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 204558us
I20260709 13:05:15.224924 21552 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.282394 21585 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:15.557528 21559 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:05:15.593845 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 383175us
I20260709 13:05:15.560370 21603 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:15.603472 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 394738us
I20260709 13:05:15.291333 21303 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.350919 20981 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:15.616983 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.599150 21553 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:15.601516 21192 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.940s	user 0.006s	sys 0.006s
I20260709 13:05:15.575122 21490 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:15.555400 21590 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:15.621976 21498 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:15.676898 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.648960 21281 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:15.620718 21509 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:15.657424 21304 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.759s	user 0.004s	sys 0.009s
I20260709 13:05:15.661589 21584 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.667721 15232 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:15.657065 21192 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:15.693907 21584 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.696041 21192 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:15.698453 15229 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:15.652978 21539 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:05:15.585350 21580 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:15.701401 15234 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:15.674170 21498 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:15.696527 21584 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:15.709026 15054 connection.cc:256] RPC call timeout handler was delayed by -1.6886038090000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 13:05:15.705907 15232 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:15.707943 21498 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:15.690405 20976 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:15.788920 15054 connection.cc:256] RPC call timeout handler was delayed by -1.1396038090000502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2631s for a response
I20260709 13:05:15.687875 21588 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:15.789402 15054 connection.cc:256] RPC call timeout handler was delayed by -1.0527786639999022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 13:05:15.706950 21539 consensus_queue.cc:1243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:15.826999 15232 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:05:15.851750 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48392.5us
W20260709 13:05:15.859777 15843 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.661s	user 0.006s	sys 0.002s
I20260709 13:05:15.860251 15843 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:15.877911 21588 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:15.885610 21615 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:05:15.887888 21608 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:15.903537 21624 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.904323 21624 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.906528 21505 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:15.909085 21546 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:15.909734 21546 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.910255 21546 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:15.912075 21509 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.792002 21490 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.916965 21490 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.919591 21490 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:15.922137 21524 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:15.922796 21524 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:15.926081 21546 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:15.930558 15844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1573) took 2353 ms (client timeout 1751 ms). Trace:
W20260709 13:05:15.931514 15844 rpcz_store.cc:269] 0709 13:05:13.577022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.577270 (+   248us) service_pool.cc:224] Handling call
0709 13:05:15.930282 (+2353012us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:15.930430 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353803008}
I20260709 13:05:15.934971 20916 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:15.935477 20916 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:15.936358 20976 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.937794 21618 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:05:15.702405 21281 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.816537 21622 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:05:15.944191 15159 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.786s	user 0.010s	sys 0.000s
I20260709 13:05:15.946094 15159 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:15.950080 21526 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:15.936267 20916 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.958113 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:15.961211 15846 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:15.961309 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.006s	sys 0.035s
W20260709 13:05:15.961807 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.006s	sys 0.035s
W20260709 13:05:15.962680 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 676471us
I20260709 13:05:15.963649 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:15.854238 21592 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:15.969594 15844 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:15.948776 21622 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:15.951370 21450 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.972886 21490 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:15.974041 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:15.971088 15053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.014s	sys 0.040s
I20260709 13:05:15.983902 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:15.936884 20976 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:16.025976 15053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.023s	sys 0.040s
I20260709 13:05:16.036332 20916 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.038261 20981 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:16.042052 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59924.9us
I20260709 13:05:16.044158 15160 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:15.966835 21603 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:16.073305 21587 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.073674 21603 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:15.986193 21614 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:16.076295 21544 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:16.073967 21587 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.079509 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1473) took 1893 ms. Trace:
I20260709 13:05:16.080338 21587 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:16.081244 21556 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 1.960s	user 0.011s	sys 0.018s
I20260709 13:05:16.082049 21552 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.079947 15839 rpcz_store.cc:276] 0709 13:05:14.186329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.279491 (+ 93162us) service_pool.cc:224] Handling call
0709 13:05:14.787009 (+507518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.798418 (+ 11409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.079290 (+1280872us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54006cdc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.079442 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166207872}
I20260709 13:05:16.053793 15844 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:16.033247 21266 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:05:16.088850 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.018s	sys 0.028s
I20260709 13:05:16.050469 15846 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:15.986513 21526 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.095574 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.095942 21526 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:16.097090 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.019s	sys 0.028s
W20260709 13:05:16.090595 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.039s	sys 0.082s
W20260709 13:05:16.097563 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.039s	sys 0.082s
W20260709 13:05:16.097764 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 770944us
I20260709 13:05:16.099045 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:16.067986 21609 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.449s	user 0.001s	sys 0.002s
W20260709 13:05:16.100087 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.001s
I20260709 13:05:16.044499 21585 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.072374 21499 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:16.108109 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.009s	sys 0.001s
I20260709 13:05:16.103551 21229 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:16.128834 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:16.142074 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 969936us
I20260709 13:05:16.142724 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2338) took 1252 ms. Trace:
I20260709 13:05:16.143218 15230 rpcz_store.cc:276] 0709 13:05:14.890444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.906818 (+ 16374us) service_pool.cc:224] Handling call
0709 13:05:14.907155 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.160575 (+253420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.142404 (+981829us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54003d9fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:16.142638 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581501312}
I20260709 13:05:16.146287 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1486) took 1436 ms. Trace:
I20260709 13:05:16.146824 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1488) took 1436 ms. Trace:
I20260709 13:05:16.146719 15163 rpcz_store.cc:276] 0709 13:05:14.709652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.711560 (+  1908us) service_pool.cc:224] Handling call
0709 13:05:14.732651 (+ 21091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.777565 (+ 44914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.777595 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.777635 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:14.777663 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.146037 (+1368374us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400378054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:16.146094 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.146228 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102858112}
I20260709 13:05:16.147271 15154 rpcz_store.cc:276] 0709 13:05:14.710289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.712372 (+  2083us) service_pool.cc:224] Handling call
0709 13:05:14.712648 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.909554 (+196906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.909595 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.909667 (+    72us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:14.909720 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.146597 (+1236877us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540017eb54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:16.146652 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.146773 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452257664}
I20260709 13:05:16.117170 21595 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.114245 21592 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.045217 21450 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:16.126963 21557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.007s	sys 0.024s
I20260709 13:05:16.114614 21499 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:16.176739 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2331) took 1360 ms. Trace:
I20260709 13:05:16.178117 15236 rpcz_store.cc:276] 0709 13:05:14.816202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.827944 (+ 11742us) service_pool.cc:224] Handling call
0709 13:05:14.909456 (+ 81512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.917400 (+  7944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.176555 (+1259155us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b540043d350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.176679 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155007872}
I20260709 13:05:16.169008 21553 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.179720 21553 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.182849 21553 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.139747 21624 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.105935 21490 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:16.139338 21229 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:16.169245 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.009s	sys 0.010s
I20260709 13:05:16.146060 21641 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:16.156996 21612 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:16.197980 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:16.170493 21281 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:16.185355 21557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.007s	sys 0.024s
I20260709 13:05:16.175621 21628 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.199373 21604 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.001s
W20260709 13:05:16.198073 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.009s	sys 0.011s
I20260709 13:05:16.157195 21614 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 13:05:16.232901 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 680187us
W20260709 13:05:16.235251 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.235481 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:16.236752 15230 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:05:16.234823 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 191827us
I20260709 13:05:16.196457 21505 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:05:16.162118 21622 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.221889 21546 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:05:16.222185 21499 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.184262 21498 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.240370 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:16.240438 15229 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:16.248217 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.264801 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:16.201530 21624 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:16.188388 21592 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.283093 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:16.293744 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:16.229094 21641 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:16.246021 21499 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:16.178201 21619 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.106s
W20260709 13:05:16.268249 21576 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.634s	user 0.008s	sys 0.011s
I20260709 13:05:16.178092 21595 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.362506 21595 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:16.363057 21595 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.363480 21595 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.346259 21619 consensus_queue.cc:1243] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:16.365288 21633 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.179s
I20260709 13:05:16.344156 21638 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:16.366218 20976 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:16.366447 21633 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:16.225033 21616 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 13:05:16.366865 21633 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:16.367373 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 73769.7us
W20260709 13:05:16.367633 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.367831 21436 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:16.368432 21436 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:16.367919 21633 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.353895 15235 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
W20260709 13:05:16.354789 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71829.9us
I20260709 13:05:16.387372 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:16.355990 21509 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.357033 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2330) took 1546 ms. Trace:
I20260709 13:05:16.390476 15234 rpcz_store.cc:276] 0709 13:05:14.810780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.815689 (+  4909us) service_pool.cc:224] Handling call
0709 13:05:16.356815 (+1541126us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b5400294854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.356974 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1951278976}
I20260709 13:05:16.360255 15237 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:16.366878 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1497) took 1234 ms. Trace:
I20260709 13:05:16.369088 21452 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:16.368990 21649 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:05:16.366237 21638 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:16.398665 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 187238us
I20260709 13:05:16.250569 21628 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:16.353740 21647 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:16.373478 15840 pending_rounds.cc:85] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 13
I20260709 13:05:16.411965 15840 pending_rounds.cc:107] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:05:16.412330 15840 raft_consensus.cc:2889] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.373193 21229 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:16.196305 21635 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.131s
I20260709 13:05:16.246577 21588 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:05:16.374651 21526 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.375380 21546 consensus_queue.cc:1243] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 17.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:16.379148 21450 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.214095 21593 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:05:16.384056 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:16.384193 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 410318us
I20260709 13:05:16.430119 15230 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:16.431802 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:16.387915 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.267424 21612 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:16.221987 21620 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.182s
W20260709 13:05:16.383363 21487 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.013s	user 0.010s	sys 0.003s
I20260709 13:05:16.237833 21604 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.376183 21524 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.385106 21624 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:16.451462 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:16.392273 15158 rpcz_store.cc:276] 0709 13:05:15.132219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.141263 (+  9044us) service_pool.cc:224] Handling call
0709 13:05:15.141598 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.141825 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:15.221230 (+ 79405us) log.cc:844] Serialized 14 byte log entry
0709 13:05:16.366406 (+1145176us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b1800493840. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287
0709 13:05:16.366511 (+   105us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:16.366569 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:16.366616 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.366690 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.366810 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28315,"prepare.queue_time_us":475864,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6123,"raft.queue_time_us":478620,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":86232,"spinlock_wait_cycles":57722112,"thread_start_us":460483,"threads_started":2}
I20260709 13:05:16.377704 21592 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.379287 21595 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:16.389103 21587 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.455580 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:16.448261 15052 connection.cc:256] RPC call timeout handler was delayed by -1.0756399299999657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3155s for a response
I20260709 13:05:16.395875 21452 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.369318 21436 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.432785 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:16.431694 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:16.412678 21638 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:16.429124 21229 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.395570 21509 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.425871 21266 consensus_queue.cc:1243] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:16.454279 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:16.422176 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:16.441386 20976 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.487481 15202 leader_election.cc:400] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19.
I20260709 13:05:16.425577 21488 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:05:16.490028 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.514256 15202 leader_election.cc:403] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:05:16.492753 15236 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:16.493270 15846 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:16.452693 21487 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:16.497215 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:16.450577 21590 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.458707 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:16.445369 21526 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.498150 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:16.499717 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:16.499993 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68385.5us
I20260709 13:05:16.496625 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:16.501328 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68632.8us
I20260709 13:05:16.501960 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.502431 15155 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:16.503424 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:16.530179 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 71760.6us
W20260709 13:05:16.530447 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 108400us
I20260709 13:05:16.530908 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:16.531677 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:16.533850 15158 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:16.534785 15161 raft_consensus.cc:2368] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19.
I20260709 13:05:16.500480 21452 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:16.508747 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.537369 15156 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:16.441557 21557 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 13:05:16.514868 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 62432.1us
I20260709 13:05:16.522215 15162 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:16.493408 21635 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:05:16.539430 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 84456.5us
W20260709 13:05:16.541141 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.554952 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.557634 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.511144 21656 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:16.558553 15163 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.510695 21603 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:16.551371 15162 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:16.522915 21498 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.488130 21587 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.567970 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.537638 21620 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:16.571456 15162 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:16.542867 21487 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 13:05:16.545159 21651 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:05:16.523840 21536 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19.
I20260709 13:05:16.577461 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.569355 21603 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:16.551254 21266 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.603956 21266 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.604141 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.604423 21266 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:16.604707 21642 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:05:16.606539 21439 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:16.606937 21439 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:16.607700 21439 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.540978 21526 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.610510 21526 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.541119 21612 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:16.613260 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:16.543282 21645 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.557505 21624 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:16.570413 21436 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.618978 21604 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.536077 21590 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.586017 21524 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.624348 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2328) took 1814 ms. Trace:
I20260709 13:05:16.625371 21645 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:16.625943 15231 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:05:16.627705 21624 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:16.628196 21645 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:16.628422 21624 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:16.629045 15843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1549) took 1539 ms (client timeout 2008 ms). Trace:
I20260709 13:05:16.629830 21624 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:16.631140 15160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2615) took 1481 ms (client timeout 1711 ms). Trace:
W20260709 13:05:16.631656 15160 rpcz_store.cc:269] 0709 13:05:15.150002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.152871 (+  2869us) service_pool.cc:224] Handling call
0709 13:05:16.630898 (+1478027us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540013ecd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.631040 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444350208}
I20260709 13:05:16.634878 21509 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.635291 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2613) took 1530 ms. Trace:
I20260709 13:05:16.521142 20976 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.573751 21595 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:16.587244 21653 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:16.641322 21595 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.643770 21595 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.587590 21617 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.133s
I20260709 13:05:16.586716 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.514858 21490 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:05:16.646103 15228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2619) took 1539 ms (client timeout 1524 ms). Trace:
I20260709 13:05:16.646509 15158 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:16.646610 15228 rpcz_store.cc:269] 0709 13:05:15.107111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.123712 (+ 16601us) service_pool.cc:224] Handling call
0709 13:05:16.645833 (+1522121us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003d9ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.646015 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313950976}
I20260709 13:05:16.648016 21490 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:16.543049 21613 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.648718 21490 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.649434 21628 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.649214 21490 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.651645 21490 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.653636 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:16.654433 15848 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:16.658478 20976 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.589154 21620 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:16.659870 21620 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.661319 21439 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.651620 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:16.629554 15843 rpcz_store.cc:269] 0709 13:05:15.089309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.089435 (+   126us) service_pool.cc:224] Handling call
0709 13:05:16.624353 (+1534918us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400730354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.628967 (+  4614us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94405504}
W20260709 13:05:16.663355 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 134318us
I20260709 13:05:16.624955 15233 rpcz_store.cc:276] 0709 13:05:14.810142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.810282 (+   140us) service_pool.cc:224] Handling call
0709 13:05:16.624070 (+1813788us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54002d3a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.624270 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299568896}
I20260709 13:05:16.667799 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2622) took 1522 ms. Trace:
W20260709 13:05:16.668366 15126 connection.cc:256] RPC call timeout handler was delayed by -1.03431010099996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2677s for a response
I20260709 13:05:16.668390 15237 rpcz_store.cc:276] 0709 13:05:15.145645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.555912 (+410267us) service_pool.cc:224] Handling call
0709 13:05:16.360037 (+804125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.391827 (+ 31790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.667503 (+275676us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b5400294850. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.667706 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848333568}
I20260709 13:05:16.683269 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:16.683633 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:16.602385 15156 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:16.603562 21603 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.614034 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:16.559319 21531 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:16.542927 21655 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:16.619963 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89171.4us
I20260709 13:05:16.623226 21590 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.635834 15235 rpcz_store.cc:276] 0709 13:05:15.105378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.150917 (+ 45539us) service_pool.cc:224] Handling call
0709 13:05:16.635071 (+1484154us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400177854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:16.635218 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2732332800}
I20260709 13:05:16.616990 21649 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.649151 21613 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.543136 21650 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.678687 21557 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.689996 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:16.690229 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:16.706780 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:16.715327 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.721410 15082 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:16.721642 15083 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:16.692165 21635 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:16.700542 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.727802 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:16.600208 15162 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:16.683513 21498 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.731599 21498 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.733960 21498 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.734459 21498 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.734930 21498 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.735373 21657 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.736093 21657 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.694456 21617 consensus_queue.cc:1243] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:05:16.706655 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:16.730715 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.700177 21655 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:16.737136 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:16.690976 21634 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.788941 14972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103420us
I20260709 13:05:16.789423 21634 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.738997 21613 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:16.739638 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:16.740324 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.694713 21628 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:05:16.806720 15847 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:16.696523 21531 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:16.741060 21660 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.160s	user 0.004s	sys 0.000s
I20260709 13:05:16.770008 21303 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.704963 21650 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:16.700335 21590 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.823819 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.824730 15085 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:16.743427 21266 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:16.693439 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 161875us
W20260709 13:05:16.759688 15051 connection.cc:256] RPC call timeout handler was delayed by -1.19365868400007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 13:05:16.749212 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.730618 15162 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.786439 15084 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.748586 21042 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.797447 15081 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:05:16.771885 21654 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.260s	user 0.000s	sys 0.000s
I20260709 13:05:16.704682 21649 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.806788 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:16.742600 21543 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:16.792685 21643 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
I20260709 13:05:16.744750 21587 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:16.829058 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.889228 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.890367 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.831861 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:16.719481 21557 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.792055 21490 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:16.750476 21664 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:05:16.832234 15083 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
W20260709 13:05:16.854595 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 305259us
W20260709 13:05:16.846426 21605 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.491s	user 0.007s	sys 0.008s
I20260709 13:05:16.832594 21042 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.825738 21650 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.833140 21590 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.825970 21498 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.750104 21638 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.900574 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:16.930728 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:16.931532 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.953169 15233 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:05:16.932196 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 242306us
I20260709 13:05:16.899983 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:16.949553 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 96068.3us
I20260709 13:05:16.782229 21635 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.957852 15084 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:16.940196 21605 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.588s	user 0.008s	sys 0.009s
I20260709 13:05:16.946256 15235 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:05:16.964911 15235 pending_rounds.cc:85] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 12
I20260709 13:05:16.946787 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:16.948175 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.815999 21452 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.943868 15158 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:16.899042 21303 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.950419 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:16.893476 21613 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.963112 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:16.902474 21634 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:16.963588 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.033547 15161 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:16.965718 15228 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:16.965870 15081 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:16.965276 15235 pending_rounds.cc:107] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:05:17.035274 15235 raft_consensus.cc:2889] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.966601 15160 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:16.966415 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:17.036742 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70514.8us
I20260709 13:05:16.893707 21665 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:05:16.967655 15203 leader_election.cc:400] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19.
I20260709 13:05:16.966595 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:16.942324 21042 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:16.951253 21499 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.004s
I20260709 13:05:16.933367 21566 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.024521 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.024273 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 334119us
I20260709 13:05:16.989414 15234 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:17.030081 15156 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:16.947016 21579 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.949846 21656 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:16.990329 21643 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:05:17.049396 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81857.3us
I20260709 13:05:17.069840 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:16.971366 15085 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:05:17.052078 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.052827 15846 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:16.988543 21572 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:16.901793 21543 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:16.989670 21612 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:17.063079 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 231289us
I20260709 13:05:17.063087 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.235028 15081 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:17.095651 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:17.235670 15085 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.042436 21593 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.081323 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.246011 15161 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:17.246270 21042 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.247012 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:17.247571 15161 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.248466 21042 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.249006 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.249915 15081 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:17.029883 21303 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.137120 21640 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.137435 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:17.138531 21667 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:05:17.138600 21641 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:17.139701 21671 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:05:17.142010 21543 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.073962 21634 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.045830 21613 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.148571 15156 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:17.150300 21439 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.151909 21557 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:17.157407 21229 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.007s
I20260709 13:05:17.076927 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:17.089449 21587 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:17.158381 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.081696 21579 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.128471 21680 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
W20260709 13:05:17.175457 21436 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.556s	user 0.003s	sys 0.005s
I20260709 13:05:17.066802 21566 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:17.178581 21677 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:05:17.301187 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.178435 21656 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:16.952145 21655 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.179219 21657 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.185678 21590 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.187939 21281 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:17.305737 21281 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:17.306978 21281 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.307830 21579 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.308300 21281 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:17.189217 21452 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.283299 21572 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.190225 21638 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.193300 21531 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:17.309278 21572 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.193456 21649 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.193974 21266 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.309836 21572 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.180526 21635 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.199250 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2620) took 2090 ms. Trace:
I20260709 13:05:17.310591 15231 rpcz_store.cc:276] 0709 13:05:15.113093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.555460 (+442367us) service_pool.cc:224] Handling call
0709 13:05:16.625531 (+1070071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.626531 (+  1000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.198895 (+572364us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002d3a50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:17.199157 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413511552}
I20260709 13:05:17.207275 15082 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:17.235487 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:17.071169 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 119883us
W20260709 13:05:17.312460 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 77052.2us
I20260709 13:05:17.312573 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:17.313009 21638 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:17.313496 15082 raft_consensus.cc:2376] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15. Re-sending same reply.
I20260709 13:05:17.313767 21638 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:17.314688 21638 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:17.093894 21664 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.315019 21638 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:17.315150 21613 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.315429 21664 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.315891 21638 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:17.242575 21603 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 13:05:17.317368 21603 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.259523 21640 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:17.320343 21640 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.321867 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.322158 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:17.241730 21686 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:17.323184 15079 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:17.324255 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.268124 15160 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:17.275806 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 198949us
I20260709 13:05:17.326614 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:17.327560 21531 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:17.327714 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:17.328255 21531 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:17.328357 21628 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.313786 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.280072 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.329978 15081 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:17.329501 21531 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.331070 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:17.332350 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.279021 21439 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.282608 21303 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:17.257469 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.294761 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.279228 21634 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.266942 21641 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:17.409577 15087 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:17.297003 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.410024 21641 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.410940 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.411273 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.299577 21650 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.290560 15162 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.414402 15845 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.378122 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.415314 15844 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:17.386092 21597 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.334990 21579 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:17.354116 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.385752 15079 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:17.330420 15084 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:17.360487 21682 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:17.362912 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.377295 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:17.428066 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:17.428823 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.376703 15081 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:17.392032 15161 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:17.384595 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 70878.4us
I20260709 13:05:17.429829 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.430428 15162 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:05:17.431107 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:17.386137 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.393927 15085 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:17.385607 21664 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.433310 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.334959 21229 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:17.394811 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 137582us
I20260709 13:05:17.397398 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:17.439875 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.406095 21628 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.406513 21603 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.408558 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.411623 15156 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:17.412264 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.305816 21587 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:17.432523 15844 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [14.13-16.14]
I20260709 13:05:17.383831 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:17.433422 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47382.5us
I20260709 13:05:17.435266 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.411432 21641 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:17.454953 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.458316 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.459797 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.445885 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:17.434139 15079 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:17.447407 15161 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:05:17.426254 21194 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.464262 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.468477 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.469465 15844 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.444140 15841 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:17.401276 21301 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.401427 21663 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:17.401065 21303 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.437841 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:17.426760 21638 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.462658 15161 pending_rounds.cc:85] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 10
I20260709 13:05:17.414399 21650 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:17.449291 21516 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:05:17.471751 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.493150 15842 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:17.493476 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:17.478235 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.481050 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.443290 21266 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:17.448863 21229 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:17.483477 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45778.7us
I20260709 13:05:17.474615 21042 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:17.448618 21657 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:17.508965 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.484671 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.448436 21617 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:05:17.490273 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.472299 15080 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:17.462634 21612 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:17.489537 21664 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.483181 21566 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.471657 21543 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:17.468247 21194 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.467653 21546 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.506840 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.484530 15161 pending_rounds.cc:107] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:05:17.526010 15161 raft_consensus.cc:2889] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:17.490067 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.504012 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.457034 21628 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.494804 15081 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:17.513298 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.517896 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.434165 21597 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.535282 15841 raft_consensus.cc:2410] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:17.520120 15084 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:17.504949 21590 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:17.511140 21657 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.508723 21042 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.530532 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:17.508384 21229 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.524773 21579 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:17.481875 21301 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:17.509433 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.590448 15081 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:17.529397 21646 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:17.593111 21439 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.593469 21646 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:17.551334 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44580.4us
I20260709 13:05:17.534605 21450 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.551851 21597 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.489699 21303 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.553325 21452 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:17.515636 21612 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.533455 15127 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.532886 21566 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:17.590561 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.615492 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.531579 21546 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.587248 21590 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:17.531733 21194 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.602300 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:17.556993 21657 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:17.561859 21042 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.554926 21543 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:17.609967 15155 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.035s	sys 0.037s
I20260709 13:05:17.601542 21646 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.611989 21612 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:17.614372 21301 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:17.624706 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 91345.3us
I20260709 13:05:17.615284 21597 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.607074 21579 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:17.654510 21579 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.658494 21572 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.668921 21452 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:17.675908 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.676113 15079 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:17.678750 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.680054 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.682109 15844 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:17.683149 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.682482 21590 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:17.683682 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.691023 21572 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.692332 15157 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.692705 15157 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:17.693472 15157 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.694583 15157 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:17.698127 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.697425 21543 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.699777 21634 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.623064 21546 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.701468 21516 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.702024 21516 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.705185 21516 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.705760 21640 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.706559 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.707688 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:17.634871 15155 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.035s	sys 0.037s
I20260709 13:05:17.641121 15839 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:17.625042 15158 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.000s	sys 0.022s
I20260709 13:05:17.633587 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.622794 21566 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.643774 15127 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:17.588186 21587 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.652170 21628 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.652786 21597 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.642812 21690 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.708811 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:17.722070 15158 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.022s
W20260709 13:05:17.723883 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 90395us
I20260709 13:05:17.718706 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3046) took 1190 ms. Trace:
I20260709 13:05:17.774437 15155 rpcz_store.cc:276] 0709 13:05:16.528636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.550114 (+ 21478us) service_pool.cc:224] Handling call
0709 13:05:16.550500 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:16.577441 (+ 26941us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:16.850430 (+272989us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.039550 (+189120us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:17.040126 (+   576us) log.cc:844] Serialized 52 byte log entry
0709 13:05:17.040261 (+   135us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:17.082109 (+ 41848us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.084982 (+  2873us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.718281 (+633299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.718391 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.718454 (+    63us) raft_consensus.cc:1685] finished
0709 13:05:17.718474 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.718649 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62121,"prepare.queue_time_us":726002,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":3350,"raft.queue_time_us":218634,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":29256,"thread_start_us":984254,"threads_started":3,"wal-append.queue_time_us":240302}
W20260709 13:05:17.717243 21639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.015s	sys 0.009s
W20260709 13:05:17.724332 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80661.4us
I20260709 13:05:17.774461 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.780887 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:17.720158 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.712976 21301 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:17.735707 21439 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:17.733703 21566 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:17.719592 21674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.088s	sys 0.065s
I20260709 13:05:17.718711 21603 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.735844 21281 raft_consensus.cc:2764] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:17.814399 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1491) took 1308 ms. Trace:
I20260709 13:05:17.828981 15839 rpcz_store.cc:276] 0709 13:05:16.506501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.506683 (+   182us) service_pool.cc:224] Handling call
0709 13:05:16.507062 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.718297 (+1211235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.812815 (+ 94518us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54006d7c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:17.814307 (+  1492us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607637888}
I20260709 13:05:17.830912 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:17.773973 21628 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.822302 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3076) took 1073 ms. Trace:
W20260709 13:05:17.807396 21639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.016s	sys 0.009s
I20260709 13:05:17.721230 21622 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
W20260709 13:05:17.823510 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 482438us
I20260709 13:05:17.818470 21516 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:17.826807 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:17.819603 21439 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.836114 15082 rpcz_store.cc:276] 0709 13:05:16.748687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.314327 (+565640us) service_pool.cc:224] Handling call
0709 13:05:17.314666 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.314903 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:17.346246 (+ 31343us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.821802 (+475556us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:17.821884 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:17.821964 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.822065 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.822225 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30272,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":122,"thread_start_us":433948,"threads_started":1}
I20260709 13:05:17.839978 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.871603 15085 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:17.845083 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 447789us
W20260709 13:05:17.846124 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.813807 21301 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.875828 21301 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:17.824627 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:17.830925 21674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.089s	sys 0.066s
I20260709 13:05:17.822603 21638 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.844591 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3075) took 1095 ms. Trace:
I20260709 13:05:17.881844 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.858757 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.883086 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.884310 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.885025 21572 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:17.885322 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.886595 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.887308 21617 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.887954 21617 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.887653 15841 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:17.889721 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.893172 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.873950 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.874191 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:17.895639 21649 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:17.833176 21664 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:17.874815 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:17.873950 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:17.902839 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:17.903335 15234 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:17.906495 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.849768 21663 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:17.921034 21663 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:17.921530 21663 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:17.834290 15157 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:17.874531 15233 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:17.924288 21597 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:17.864235 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.924897 21597 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:17.924950 15081 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:17.925431 21597 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:17.925822 21597 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:17.926084 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3052) took 1337 ms. Trace:
I20260709 13:05:17.927827 21042 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:17.928968 21619 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:05:17.903779 21450 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:17.931767 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:17.932139 21303 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.932987 15156 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.932542 15157 rpcz_store.cc:276] 0709 13:05:16.588460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.605268 (+ 16808us) service_pool.cc:224] Handling call
0709 13:05:16.605582 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.692242 (+1086660us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000972a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f95bdc0 00007f0d3f9e61c0 00007f0d3f9e7231 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c
0709 13:05:17.925826 (+233584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.926031 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404982528}
I20260709 13:05:17.911158 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.913465 21633 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.843824 21603 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.929013 15163 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:17.891089 21617 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.928256 21612 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.870306 21487 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:17.945684 15201 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.959652 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:17.938431 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.963291 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:17.899550 21439 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:17.904803 21516 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:17.982426 21516 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:17.849437 21628 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.986711 21628 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.987382 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:17.967553 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:17.968204 15082 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:17.970057 21641 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
I20260709 13:05:17.969655 21663 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.972064 21612 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:17.973408 21620 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.973860 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:17.974551 15162 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:17.975375 21657 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:17.976370 15234 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.976948 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.007097 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.007266 15235 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 10.
I20260709 13:05:17.960528 21303 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:17.977865 15237 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:17.977954 21603 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.978559 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:17.979758 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.009719 15232 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:17.980733 15229 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:17.966809 15839 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:17.993310 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.002640 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:17.960011 21281 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.939045 21572 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:05:18.009884 15230 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:17.999541 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:18.029544 15840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.012s
W20260709 13:05:18.029855 15840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.004s	sys 0.012s
I20260709 13:05:18.016376 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:18.017426 21657 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:17.956182 21301 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.026360 21597 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.013448 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.030483 15843 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:18.009214 15200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.074s	sys 0.079s
I20260709 13:05:18.031457 15236 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:18.023294 21646 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.044827 15200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.075s	sys 0.079s
W20260709 13:05:18.036499 15847 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.006s	sys 0.013s
I20260709 13:05:18.037776 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:18.046355 15847 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.013s
I20260709 13:05:18.047142 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.048525 15080 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:18.051676 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.015413 21690 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:18.052434 15156 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:18.052783 15230 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:18.000195 21452 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:05:18.028865 21487 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:18.053669 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.055173 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:18.055540 15236 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:18.055776 15843 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.056141 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:18.039234 21303 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:18.058346 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.056227 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.058166 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:18.059774 15846 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:18.040639 21531 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:18.061364 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.062608 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.007781 21229 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:18.064687 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.075178 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.076366 15163 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:18.072893 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.077487 15234 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.077343 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.078284 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.078878 15085 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:18.079352 15160 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:18.082633 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.085784 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.089246 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.090248 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.092217 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.093015 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.094249 15230 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:18.094527 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.097429 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.098228 15156 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:18.099268 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.101475 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.092943 15229 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:18.102090 15236 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:18.102352 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.100471 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.103539 15156 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:18.102169 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.107326 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.081341 21487 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:18.109014 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.109149 21646 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:18.081700 21612 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:18.110661 21612 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:18.111166 21612 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.111886 15843 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:18.113078 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:18.114041 21487 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:17.881690 15086 rpcz_store.cc:276] 0709 13:05:16.748426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.274939 (+526513us) service_pool.cc:224] Handling call
0709 13:05:17.390917 (+115978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.391126 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:17.434124 (+ 42998us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.843254 (+409130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:17.843804 (+   550us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:17.843867 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.843953 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.844156 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37993,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":739,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11413,"thread_start_us":407950,"threads_started":1}
I20260709 13:05:18.086313 21303 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.117251 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.117944 15081 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:18.118820 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.119527 15081 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:18.120265 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.120960 21614 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:18.121382 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.122064 15082 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.117285 21303 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.124347 21303 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.125608 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.125888 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.126502 15162 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.127002 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.127763 15085 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:18.130322 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.131525 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:18.138320 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.115872 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.132542 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.139958 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.128742 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.142006 15085 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:18.145249 15081 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 13: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:18.077389 21641 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:18.147653 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.115872 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:18.148777 15234 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:18.147589 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.088658 15839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:18.157661 15839 raft_consensus.cc:606] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:18.161373 15839 raft_consensus.cc:614] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:18.160714 15841 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:18.165084 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.165797 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.112995 21531 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.168008 15082 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:18.168291 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.169170 15082 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 15. Ops: [13.12-13.12]
I20260709 13:05:18.151103 15162 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:18.172076 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:18.172943 21633 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:18.173442 21633 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.173621 21597 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.174377 21633 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.153036 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.176121 15163 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:18.055634 21603 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:05:18.087826 21301 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.095366 21638 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.198048 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.146157 21690 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:18.201846 15086 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:18.153729 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.148658 21656 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:18.205399 15079 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:05:18.205847 15079 pending_rounds.cc:85] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 13
I20260709 13:05:18.206040 21632 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:18.206466 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3045) took 1691 ms. Trace:
I20260709 13:05:18.207841 21632 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:18.208297 21632 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.166980 21617 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:18.206070 15079 pending_rounds.cc:107] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:05:18.166776 21588 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.210029 15079 raft_consensus.cc:2889] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.179570 15085 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:18.180513 15082 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:18.186532 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:18.215631 15082 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:18.187522 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.217304 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.217263 15082 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:18.218189 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.219345 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.189440 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.191151 15156 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:18.154677 21619 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:18.092286 21194 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.198800 15162 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.194406 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.152417 21628 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:05:18.175396 21229 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:18.224669 15160 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:18.151926 21593 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:18.202975 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.203877 21624 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:18.204331 15846 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:05:18.206969 15159 rpcz_store.cc:276] 0709 13:05:16.515089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.542704 (+ 27615us) service_pool.cc:224] Handling call
0709 13:05:16.543139 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.570890 (+ 27751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.570941 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.571515 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:05:17.244261 (+672746us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b180042ae40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9
0709 13:05:17.387043 (+142782us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180042ae40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:17.828690 (+441647us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:17.856921 (+ 28231us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.867640 (+ 10719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.867742 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.206204 (+338462us) raft_consensus.cc:1685] finished
0709 13:05:18.206230 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.206398 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37903,"prepare.queue_time_us":27141,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":59325,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":10601,"spinlock_wait_cycles":10045696,"thread_start_us":441336,"threads_started":1,"wal-append.queue_time_us":1616538}
I20260709 13:05:18.220697 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.229677 21649 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:18.164563 21487 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:18.232658 15841 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:05:18.160234 21641 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:18.237864 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.246357 15160 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:18.240164 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.240500 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.248467 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.251168 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.255009 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:18.256950 15159 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 13:05:18.258401 15159 raft_consensus.cc:1217] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Deduplicated request from leader. Original: 9.7->[11.8-15.9]   Dedup: 11.8->[15.9-15.9]
I20260709 13:05:18.259011 15159 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: [11.8-15.9]
I20260709 13:05:18.221912 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:18.241904 15126 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.574s (SENT)
I20260709 13:05:18.245412 15841 pending_rounds.cc:85] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 11
I20260709 13:05:18.260872 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.211345 21614 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.261508 15841 pending_rounds.cc:107] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:05:18.275207 15841 raft_consensus.cc:2889] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.276988 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:18.222985 21656 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:18.279320 21682 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:18.279707 15842 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:18.279857 21682 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:18.280493 21612 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.280689 21682 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.272266 15081 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:18.273530 15846 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:05:18.279554 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57748us
I20260709 13:05:18.281142 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:18.286006 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44137.3us
I20260709 13:05:18.288322 15085 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:18.269587 15163 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:18.281273 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:18.294601 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.301733 15082 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 13:05:18.305084 15234 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:18.270107 15162 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:18.230460 21655 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:18.190490 21531 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.332310 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:18.333709 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 50517.4us
W20260709 13:05:18.343546 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.248998 21579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.013s	sys 0.047s
W20260709 13:05:18.346576 21581 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.008s	sys 0.022s
W20260709 13:05:18.347147 21581 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.008s	sys 0.022s
W20260709 13:05:18.324460 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.007s	sys 0.028s
I20260709 13:05:18.234591 21649 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:18.348402 21649 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:18.349102 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.008s	sys 0.028s
W20260709 13:05:18.350566 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 966042us
I20260709 13:05:18.253556 21450 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:18.245157 21487 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.240880 21042 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.351401 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:18.370234 15081 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:18.279434 21194 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:18.362658 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:18.346829 21579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.013s	sys 0.047s
I20260709 13:05:18.261219 21593 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:18.375986 15080 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:18.377930 15228 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.360405 15160 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:18.281734 21663 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:18.390702 21663 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:18.382179 15842 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:18.389772 21705 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:18.391335 21699 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:05:18.384172 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:18.394460 21655 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:18.325896 21657 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:18.389917 21702 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:05:18.252048 21505 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 13:05:18.324942 21624 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:18.397444 15845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.039s	sys 0.049s
W20260709 13:05:18.429844 15845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.039s	sys 0.050s
I20260709 13:05:18.399834 15159 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:05:18.361898 21619 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.230597 21588 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.385213 15158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.002s	sys 0.019s
W20260709 13:05:18.437340 15158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.003s	sys 0.019s
I20260709 13:05:18.438226 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:18.362106 21634 raft_consensus.cc:2764] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:05:18.374065 21487 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:18.415266 15813 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.012s	sys 0.012s
W20260709 13:05:18.459297 15813 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.013s	sys 0.013s
I20260709 13:05:18.414742 21705 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:18.459843 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 643798us
I20260709 13:05:18.418349 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:18.384320 21682 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.423323 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.467113 15228 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:18.466809 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.484948 15842 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.425241 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:18.430675 15159 pending_rounds.cc:85] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 12
I20260709 13:05:18.490692 15159 pending_rounds.cc:107] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:05:18.491031 15159 raft_consensus.cc:2889] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.259775 21632 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.320284 21614 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:18.463574 21634 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.465485 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1560) took 1013 ms. Trace:
I20260709 13:05:18.508380 15845 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:18.508111 15840 rpcz_store.cc:276] 0709 13:05:17.452537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.462966 (+ 10429us) service_pool.cc:224] Handling call
0709 13:05:17.463317 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.479872 (+ 16555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.479925 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.480554 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.030917 (+550363us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:18.030956 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.031024 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.465259 (+434235us) raft_consensus.cc:1685] finished
0709 13:05:18.465285 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.465425 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":15305,"thread_start_us":509084,"threads_started":1,"wal-append.queue_time_us":951663}
I20260709 13:05:18.510726 15840 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:18.517895 15841 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:18.429459 21229 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:18.520666 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.430577 21655 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:18.522794 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.460743 21587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.014s
I20260709 13:05:18.524863 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.531222 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:18.535652 21664 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:18.536376 21664 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.536103 21593 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.540283 21641 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:18.541384 21566 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.000s
I20260709 13:05:18.541848 21566 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.544127 21566 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.544931 15231 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:18.522862 21697 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:05:18.533959 21587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.008s	sys 0.014s
I20260709 13:05:18.465387 21705 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.519898 21579 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:18.554767 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3060) took 1849 ms. Trace:
W20260709 13:05:18.551667 21557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.018s	sys 0.036s
I20260709 13:05:18.522580 21634 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.554917 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.025s
I20260709 13:05:18.524494 21531 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.522099 21229 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.550910 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:18.493414 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68267.3us
W20260709 13:05:18.558313 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.021s
W20260709 13:05:18.559561 15083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.027s	sys 0.030s
I20260709 13:05:18.560905 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.559218 15088 rpcz_store.cc:276] 0709 13:05:16.705355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.719717 (+ 14362us) service_pool.cc:224] Handling call
0709 13:05:16.741557 (+ 21840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.791911 (+ 50354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.791945 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.792417 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:05:17.400377 (+607960us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b180033d600. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:17.824214 (+423837us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:17.824255 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.145877 (+321622us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b540014da54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:18.146884 (+  1007us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.554450 (+407566us) raft_consensus.cc:1685] finished
0709 13:05:18.554482 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.554688 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57784,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":17939,"spinlock_wait_cycles":119179776,"thread_start_us":423646,"threads_started":1,"wal-append.queue_time_us":1342939}
W20260709 13:05:18.559747 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.000s	sys 0.025s
W20260709 13:05:18.562192 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.058s
I20260709 13:05:18.546115 21588 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:18.568763 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.058s
W20260709 13:05:18.568877 15083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.028s	sys 0.031s
I20260709 13:05:18.575923 15229 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 17. Ops: []
I20260709 13:05:18.576817 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.577960 15230 raft_consensus.cc:2410] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 13:05:18.567950 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.022s
I20260709 13:05:18.459263 21619 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.582330 15155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.009s	sys 0.013s
I20260709 13:05:18.535697 21614 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:18.562083 21557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.018s	sys 0.037s
W20260709 13:05:18.577052 21543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.016s	sys 0.008s
I20260709 13:05:18.567884 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:18.591975 15847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.025s	sys 0.013s
W20260709 13:05:18.574843 21623 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.005s	sys 0.026s
I20260709 13:05:18.583946 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:18.595410 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.016s	sys 0.008s
I20260709 13:05:18.591094 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:18.588269 15848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.001s	sys 0.031s
W20260709 13:05:18.594425 15847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.026s	sys 0.013s
W20260709 13:05:18.597721 15848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.001s	sys 0.032s
I20260709 13:05:18.597851 15232 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:18.598250 15230 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:18.598490 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1498) took 1188 ms. Trace:
W20260709 13:05:18.599120 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.016s	sys 0.008s
I20260709 13:05:18.589972 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:18.600936 15235 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:18.585125 21266 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:18.562400 21680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.006s	sys 0.015s
I20260709 13:05:18.594383 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.591025 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.579347 21229 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.588527 21579 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:18.585314 21590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.005s	sys 0.017s
I20260709 13:05:18.602152 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:18.628973 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 294260us
W20260709 13:05:18.596563 21623 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.005s	sys 0.027s
I20260709 13:05:18.586382 21612 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:18.595178 21543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.017s	sys 0.008s
I20260709 13:05:18.611423 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:18.607975 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.639094 15229 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:18.594655 21531 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 13:05:18.582921 15155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.009s	sys 0.013s
I20260709 13:05:18.632036 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:18.642645 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 310439us
I20260709 13:05:18.598873 15847 rpcz_store.cc:276] 0709 13:05:17.409966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.413538 (+  3572us) service_pool.cc:224] Handling call
0709 13:05:17.413865 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.414094 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:18.048597 (+634503us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.598175 (+549578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.598225 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:18.598269 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.598335 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.598439 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":674045,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":1183389,"threads_started":2}
W20260709 13:05:18.645162 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49197us
W20260709 13:05:18.609351 21516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.001s	sys 0.020s
I20260709 13:05:18.615243 21715 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.603447 21452 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.650260 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.652738 15841 raft_consensus.cc:606] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:18.617698 21680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.006s	sys 0.015s
I20260709 13:05:18.634610 21640 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:18.697098 21640 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.698360 21640 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.699697 21640 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.657131 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.700594 15081 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:18.701572 21718 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:05:18.653203 21516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.020s
I20260709 13:05:18.600371 21566 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:05:18.661090 15846 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:05:18.706192 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 56019.5us
I20260709 13:05:18.644434 21488 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:18.669100 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.669651 15161 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.656224 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.710178 15088 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:18.674696 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.711100 15236 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:18.622074 21303 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:05:18.675132 21634 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:18.676072 21546 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.975s	user 0.006s	sys 0.006s
I20260709 13:05:18.675980 21619 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:18.676966 21714 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:18.731482 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.668790 21301 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:18.668200 21452 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.678186 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2429) took 1357 ms. Trace:
W20260709 13:05:18.679252 21620 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.000s
W20260709 13:05:18.670182 21439 consensus_meta.cc:287] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.000s
I20260709 13:05:18.679342 21664 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.681329 21713 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:05:18.681865 21680 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:18.683444 21603 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:18.684258 21716 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:18.687038 21649 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.740567 20870 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:18.641680 21656 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:18.695953 21646 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:18.625502 21590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.006s	sys 0.018s
I20260709 13:05:18.648408 21703 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:18.718794 15163 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.719671 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1502) took 1686 ms. Trace:
I20260709 13:05:18.813115 15087 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 16. Ops: []
I20260709 13:05:18.813275 15158 rpcz_store.cc:276] 0709 13:05:17.032998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.033271 (+   273us) service_pool.cc:224] Handling call
0709 13:05:17.033581 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.033795 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:17.773440 (+739645us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.719204 (+945764us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.719362 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.719403 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:18.719450 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.719515 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.719616 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":304102,"prepare.queue_time_us":153634,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":284446,"spinlock_wait_cycles":26267392,"thread_start_us":1355645,"threads_started":2,"wal-append.queue_time_us":820276}
W20260709 13:05:18.794237 21718 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:05:18.721211 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.723330 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.648624 21711 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:05:18.662640 21715 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:18.725890 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.714540 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:18.821437 15079 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:18.732106 15235 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.730533 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.706404 21638 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:18.738926 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.823266 15232 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.731024 15081 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:18.740183 15083 rpcz_store.cc:276] 0709 13:05:17.320298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.525558 (+205260us) service_pool.cc:224] Handling call
0709 13:05:17.526026 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.534675 (+  8649us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:17.538366 (+  3691us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.710274 (+171908us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b18003a9a80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1
0709 13:05:17.711713 (+  1439us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b18003a9a80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:17.975447 (+263734us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003a9a80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:05:18.575576 (+600129us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.576383 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.576584 (+   201us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:18.576890 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.582023 (+  5133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.582126 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.677961 (+ 95835us) raft_consensus.cc:1685] finished
0709 13:05:18.677984 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.678126 (+   142us) inbound_call.cc:177] Queueing success response
0709 13:05:18.698322 (+ 20196us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b18003a9a80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
Metrics: {"mutex_wait_us":7686,"prepare.queue_time_us":20194,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":2795,"raft.queue_time_us":12337,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":42281,"spinlock_wait_cycles":30272256,"thread_start_us":599762,"threads_started":1,"wal-append.queue_time_us":1087468}
I20260709 13:05:18.716185 21587 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:18.744035 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.791676 21603 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.724470 21614 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:18.843935 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:18.848453 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.849835 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.779700 15844 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:18.788636 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.619963 21655 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.743945 21452 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.803531 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:18.720106 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 278938us
I20260709 13:05:18.870174 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:18.870918 15840 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:18.818974 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1508) took 1037 ms. Trace:
I20260709 13:05:18.773103 21439 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.819042 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 224796us
I20260709 13:05:18.878088 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.812767 21646 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.880262 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:18.836238 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 105776us
I20260709 13:05:18.837852 15088 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 22.
I20260709 13:05:18.745323 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.816960 21703 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.834935 21715 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.872488 15842 rpcz_store.cc:276] 0709 13:05:17.781947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.517865 (+735918us) service_pool.cc:224] Handling call
0709 13:05:18.518166 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.518361 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:18.546859 (+ 28498us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.818457 (+271598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.818535 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:18.818596 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.818706 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.818895 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35611,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":1346,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":16020,"thread_start_us":271092,"threads_started":1}
I20260709 13:05:18.884232 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:18.913228 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.916201 15158 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:18.897724 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.920724 15846 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:18.922003 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.908592 15232 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:18.732069 21546 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:18.872038 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 224224us
W20260709 13:05:18.875806 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 227354us
I20260709 13:05:18.942914 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1566) took 1048 ms. Trace:
I20260709 13:05:18.943569 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:18.943527 15080 rpcz_store.cc:276] 0709 13:05:17.894881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.230001 (+335120us) service_pool.cc:224] Handling call
0709 13:05:18.230406 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.384717 (+154311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.942609 (+557892us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540014e454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:18.942837 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333719808}
I20260709 13:05:18.822237 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:18.882092 15156 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.948197 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:18.883546 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.926690 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:18.872153 21634 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:05:18.853070 21581 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.921335 15846 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:18.962291 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:18.865867 21657 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:18.928293 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1562) took 1475 ms. Trace:
W20260709 13:05:18.865760 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 262973us
I20260709 13:05:18.929348 21617 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:18.739445 21579 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.893036 21703 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.891527 21505 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.946794 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:18.960040 15156 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:19.048573 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:18.965410 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 81960.7us
I20260709 13:05:18.936481 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:19.050697 15229 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:19.050865 15813 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:18.973667 21439 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.966853 21450 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:05:18.871946 21614 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.976641 21581 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.079704 15156 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:18.977092 15201 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.967549 15846 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:18.969867 15848 rpcz_store.cc:276] 0709 13:05:17.453330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.463972 (+ 10642us) service_pool.cc:224] Handling call
0709 13:05:17.464300 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:17.492486 (+ 28186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.492536 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:17.507121 (+ 14585us) log.cc:844] Serialized 52 byte log entry
0709 13:05:17.960900 (+453779us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:18.598363 (+637463us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.927826 (+329463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.927939 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.928006 (+    67us) raft_consensus.cc:1685] finished
0709 13:05:18.928027 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.928218 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5752,"prepare.queue_time_us":155987,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":54191,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10875,"spinlock_wait_cycles":22984576,"thread_start_us":1305437,"threads_started":3,"wal-append.queue_time_us":841740}
I20260709 13:05:18.978482 21649 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.980127 21663 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:18.980608 21488 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.982028 21597 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:18.915557 21042 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:05:18.984912 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1586) took 1357 ms. Trace:
I20260709 13:05:19.109920 15844 rpcz_store.cc:276] 0709 13:05:17.627110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.719852 (+ 92742us) service_pool.cc:224] Handling call
0709 13:05:18.984669 (+1264817us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:18.984818 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2248279296}
I20260709 13:05:18.986475 21624 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.986704 21194 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:18.987586 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1518) took 1829 ms. Trace:
I20260709 13:05:19.118965 15161 rpcz_store.cc:276] 0709 13:05:17.157882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.554996 (+397114us) service_pool.cc:224] Handling call
0709 13:05:17.555373 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.709799 (+1154426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.987290 (+277491us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400378054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:18.987507 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2562175360}
I20260709 13:05:18.988440 21620 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:18.989843 21546 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:18.993767 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:19.073982 21711 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:18.994710 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:18.977047 21301 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 13:05:18.997562 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2630) took 1042 ms. Trace:
I20260709 13:05:19.129344 15163 rpcz_store.cc:276] 0709 13:05:17.955426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.290482 (+335056us) service_pool.cc:224] Handling call
0709 13:05:18.290895 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.812908 (+522013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.997256 (+184348us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54001266d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:18.997487 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976203520}
I20260709 13:05:19.131462 15163 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:05:18.998304 21725 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:05:19.002300 21738 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
I20260709 13:05:19.002166 15235 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:19.003947 21721 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:05:19.004827 21726 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:05:19.005895 21632 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:19.015760 21742 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:05:19.016198 21266 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:19.133806 21649 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:18.927366 21739 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.031267 21634 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:18.966584 21657 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:19.037645 15154 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.993815 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2352) took 1020 ms. Trace:
I20260709 13:05:19.043038 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:19.056329 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 89385.9us
I20260709 13:05:19.046150 15842 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
W20260709 13:05:19.058506 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 412112us
I20260709 13:05:18.940151 21638 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:05:19.081548 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 104526us
I20260709 13:05:19.174423 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:18.953733 21543 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:19.064520 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:19.082458 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 429048us
I20260709 13:05:19.086531 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:19.171909 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:19.175724 15236 raft_consensus.cc:2393] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:19.170612 15231 rpcz_store.cc:276] 0709 13:05:17.973448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.976187 (+  2739us) service_pool.cc:224] Handling call
0709 13:05:17.976500 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.545411 (+568911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.993491 (+448080us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400176954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:18.993745 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1306606464}
I20260709 13:05:19.179847 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.181141 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.181746 15083 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:19.098809 15847 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:19.104984 15846 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:19.111194 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1507) took 1625 ms. Trace:
W20260709 13:05:19.076709 21736 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:05:19.186528 15079 raft_consensus.cc:2393] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:19.115190 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.047571 21617 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:19.117234 15848 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:19.191179 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.058929 21590 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:18.996284 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:19.105698 21488 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:19.106269 21566 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:19.164631 21664 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:19.157192 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:19.212329 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55228.8us
I20260709 13:05:19.075055 21505 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.023766 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1531) took 1138 ms. Trace:
I20260709 13:05:19.159484 15158 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:19.075755 21646 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.121819 21229 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.174049 21638 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:19.147631 21711 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:19.183565 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.230839 15088 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:19.231873 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.232494 15088 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
W20260709 13:05:19.114427 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.184141 15843 rpcz_store.cc:276] 0709 13:05:17.485589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.112594 (+627005us) service_pool.cc:224] Handling call
0709 13:05:18.138626 (+ 26032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.138856 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:18.176448 (+ 37592us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.646274 (+469826us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.110533 (+464259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.110606 (+    73us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:19.110670 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.110757 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.111143 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23877,"prepare.queue_time_us":25030,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":73105,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":206,"thread_start_us":909112,"threads_started":2,"wal-append.queue_time_us":692248}
I20260709 13:05:19.123741 21624 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.189678 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:19.235090 15847 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:19.235476 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:19.194023 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 468187us
I20260709 13:05:19.238000 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:19.165786 21705 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:05:19.199770 15810 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:19.192540 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:19.200358 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 135927us
I20260709 13:05:19.250849 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:19.199517 21758 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.203831 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.162336 21649 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:18.961330 21603 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.186487 21543 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:19.198518 21617 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:19.260499 21613 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:19.209076 21194 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:19.253476 21439 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:19.207973 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:19.220615 15840 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:19.174657 21657 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:19.174252 20870 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:19.222566 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1527) took 1521 ms. Trace:
I20260709 13:05:19.290493 15127 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:19.292021 15155 rpcz_store.cc:276] 0709 13:05:17.700927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.838228 (+137301us) service_pool.cc:224] Handling call
0709 13:05:17.838836 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.866597 (+ 27761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.866636 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.867251 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.648329 (+781078us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:18.648372 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.648444 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.222359 (+573915us) raft_consensus.cc:1685] finished
0709 13:05:19.222380 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.222511 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38385,"raft.queue_time_us":28490,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"thread_start_us":691111,"threads_started":1,"wal-append.queue_time_us":857213}
I20260709 13:05:19.222424 15087 rpcz_store.cc:276] 0709 13:05:17.885351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.885488 (+   137us) service_pool.cc:224] Handling call
0709 13:05:18.555359 (+669871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.817542 (+262183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.021984 (+204442us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540014da54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:19.023478 (+  1494us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540014da50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:19.023683 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124817024}
I20260709 13:05:19.186308 21266 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:19.202261 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:19.254513 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54810.5us
I20260709 13:05:19.209383 21590 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.209717 21488 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:19.248371 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1530) took 1317 ms. Trace:
I20260709 13:05:19.341741 15157 rpcz_store.cc:276] 0709 13:05:17.930636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.935398 (+  4762us) service_pool.cc:224] Handling call
0709 13:05:17.935745 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.935956 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.935981 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.936410 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.580688 (+644278us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:18.580743 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.580832 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.248092 (+667260us) raft_consensus.cc:1685] finished
0709 13:05:19.248129 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.248299 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":643973,"threads_started":1,"wal-append.queue_time_us":869561}
W20260709 13:05:19.233529 21487 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.001s
I20260709 13:05:19.255879 21612 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:05:19.170370 21739 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:19.274467 21735 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:05:19.271999 15234 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:19.272300 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1580) took 1655 ms. Trace:
I20260709 13:05:19.286019 15161 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:19.275954 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2633) took 1319 ms. Trace:
I20260709 13:05:19.252256 21680 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:19.246187 21566 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:19.298081 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.421687 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.309451 15847 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:19.271261 21744 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.255s	user 0.001s	sys 0.001s
I20260709 13:05:19.347105 15156 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:05:19.223376 21620 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:05:19.353829 15155 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:19.278671 21745 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 13:05:19.278314 21759 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:05:19.225211 21516 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.357234 15162 rpcz_store.cc:276] 0709 13:05:17.956048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.307202 (+351154us) service_pool.cc:224] Handling call
0709 13:05:18.307498 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.044175 (+736677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.044224 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.044286 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:19.044341 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.275656 (+231315us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540033e854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:19.275722 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.275881 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210641,"raft.queue_time_us":188657,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2208,"spinlock_wait_cycles":524813568,"thread_start_us":322839,"threads_started":1}
I20260709 13:05:19.457975 15162 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:19.302954 21747 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:05:19.279213 21743 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:05:19.301553 21734 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 13:05:19.356964 15839 rpcz_store.cc:276] 0709 13:05:17.616914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.199898 (+582984us) service_pool.cc:224] Handling call
0709 13:05:18.231866 (+ 31968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.232226 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.232266 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.238823 (+  6557us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.739511 (+500688us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:18.739555 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.739633 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.272092 (+532459us) raft_consensus.cc:1685] finished
0709 13:05:19.272112 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.272248 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2224,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":372870,"threads_started":1,"wal-append.queue_time_us":562827}
I20260709 13:05:19.365893 21588 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:19.370154 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1529) took 1530 ms. Trace:
I20260709 13:05:19.224829 21752 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.142s
I20260709 13:05:19.358843 21762 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:19.370518 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3064) took 1329 ms. Trace:
I20260709 13:05:19.472043 15158 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:19.470860 15160 rpcz_store.cc:276] 0709 13:05:18.041436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.387471 (+346035us) service_pool.cc:224] Handling call
0709 13:05:18.387946 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.702760 (+314814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.702805 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.703368 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.090822 (+387454us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:19.090868 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.090978 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.370313 (+279335us) raft_consensus.cc:1685] finished
0709 13:05:19.370337 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.370463 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320415,"raft.queue_time_us":316661,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10983,"thread_start_us":386897,"threads_started":1,"wal-append.queue_time_us":547712}
I20260709 13:05:19.472959 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:19.473665 15160 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:05:19.373401 21755 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:05:19.359280 21487 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.284956 21758 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.374622 21663 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.375173 21766 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:19.271349 21751 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
W20260709 13:05:19.375638 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 177905us
I20260709 13:05:19.375972 21634 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:19.376497 21764 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:19.349172 21690 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:19.378239 21740 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:19.382417 15157 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:19.277249 21194 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:19.386323 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1548) took 1496 ms. Trace:
W20260709 13:05:19.388273 21640 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.003s
I20260709 13:05:19.389554 21581 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.270678 21505 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.271739 21543 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:19.269073 21649 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.357831 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:19.392644 21614 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:19.394021 21624 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.265460 21756 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.271711 21229 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:19.277460 21439 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:19.398343 21590 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:19.269052 21750 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.132s
W20260709 13:05:19.402077 21682 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.894s	user 0.012s	sys 0.000s
I20260709 13:05:19.269140 21633 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:05:19.273571 21711 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.507998 21690 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:19.356129 21749 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:05:19.404102 21617 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.560585 20870 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:05:19.570472 21617 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:19.359406 21638 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.571642 21638 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:19.572633 21715 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.000s
W20260709 13:05:19.573383 21769 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:05:19.573160 21715 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.257604 21664 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:05:19.575440 21715 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:19.299930 21646 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:19.576081 21760 consensus_queue.cc:1175] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d45101e0851484690b4f03a4dcdb63d" responder_term: 17 status { last_received { term: 17 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:19.578135 21487 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.579689 21640 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:19.580145 21640 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:19.349828 21613 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:05:19.581022 21640 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.581952 21758 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:19.269830 21741 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:19.365454 21266 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.583693 15229 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:19.429136 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:19.429713 15080 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:19.438781 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1442) took 1011 ms. Trace:
I20260709 13:05:19.585827 15083 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:19.585745 15234 rpcz_store.cc:276] 0709 13:05:18.427912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.429109 (+  1197us) service_pool.cc:224] Handling call
0709 13:05:19.438480 (+1009371us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400058cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:19.438691 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811365632}
I20260709 13:05:19.462692 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.463938 15154 rpcz_store.cc:276] 0709 13:05:17.840264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.847381 (+  7117us) service_pool.cc:224] Handling call
0709 13:05:17.847648 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.170365 (+1322717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.369899 (+199534us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540037f854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:19.370090 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2736091008}
I20260709 13:05:19.464564 15162 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 13:05:19.590035 21739 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.485177 21762 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.592686 21762 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:19.593034 21690 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.557837 21614 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:19.596041 14983 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:19.512754 21703 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:19.491277 21766 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.597752 14983 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 11 to 14, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 14 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:19.430974 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:19.600613 15156 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:19.482254 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:19.484543 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 287348us
I20260709 13:05:19.600737 15230 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:19.433406 21680 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:19.495088 15084 rpcz_store.cc:276] 0709 13:05:17.890262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.183620 (+293358us) service_pool.cc:224] Handling call
0709 13:05:18.213852 (+ 30232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.429692 (+215840us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:18.429993 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.853120 (+423127us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.187223 (+334103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.187274 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:19.187317 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.386106 (+198789us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003efdd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:19.386152 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.386267 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34844,"prepare.queue_time_us":5495,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":279180,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":105176,"spinlock_wait_cycles":495791488,"thread_start_us":721520,"threads_started":2,"wal-append.queue_time_us":718955}
I20260709 13:05:19.505302 15088 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:05:19.488307 21452 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.005s
W20260709 13:05:19.518862 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 217150us
I20260709 13:05:19.526584 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3068) took 1467 ms. Trace:
I20260709 13:05:19.503039 21740 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.491408 21634 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:19.481978 21623 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:05:19.541699 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 183919us
W20260709 13:05:19.541776 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 129643us
W20260709 13:05:19.554160 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 315120us
I20260709 13:05:19.349185 21557 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:19.532805 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 230763us
I20260709 13:05:19.530267 21763 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.490909 21663 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.256222 21705 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.601986 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:19.613214 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:19.614333 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 132156us
I20260709 13:05:19.559085 21281 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.613075 15159 rpcz_store.cc:276] 0709 13:05:18.059003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.673149 (+614146us) service_pool.cc:224] Handling call
0709 13:05:18.673518 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.727401 (+ 53883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.727445 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.728013 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.821524 (+ 93511us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:18.821563 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.821630 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.526351 (+704721us) raft_consensus.cc:1685] finished
0709 13:05:19.526375 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.526518 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70532,"raft.queue_time_us":54582,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5513,"thread_start_us":93204,"threads_started":1,"wal-append.queue_time_us":386118}
I20260709 13:05:19.615224 14983 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.617120 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.650307 15080 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:19.664201 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:19.665619 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:19.721822 14972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56306.9us
I20260709 13:05:19.605511 21042 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.666420 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:19.661861 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:19.674711 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:19.678977 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.682071 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:19.599700 21741 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:19.599244 21766 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:19.687847 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.724131 15082 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:19.729168 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:19.653391 21773 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:05:19.805557 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 143806us
I20260709 13:05:19.731266 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:19.608091 21614 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:19.800469 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:19.803308 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 139222us
I20260709 13:05:19.807986 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:19.809024 15235 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:19.775105 14981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 358) took 27872 ms (client timeout 15000 ms). Trace:
I20260709 13:05:19.806079 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.803864 15231 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:19.799072 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:19.809715 15084 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:19.808557 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:19.815383 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.816107 15082 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:19.655047 21634 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:19.775430 14983 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:19.825668 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:19.683436 21620 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:19.826295 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:19.830338 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100011us
I20260709 13:05:19.835224 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:19.661850 21762 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:19.612442 21452 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.830976 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:19.650337 21740 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:19.805202 21516 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:19.837133 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:19.799952 21450 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.847784 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.684079 21664 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:19.809924 14981 rpcz_store.cc:269] 0709 13:04:51.845547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.845828 (+   281us) service_pool.cc:224] Handling call
0709 13:05:19.717802 (+27871974us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:13.813007 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:13.813718 (+   711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:13.813765 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:05:13.813782 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:14.007575 (+193793us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:14.007686 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:14.007719 (+    33us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:05:14.106735 (+ 99016us) tablet.cc:701] Row locks acquired
0709 13:05:14.124659 (+ 17924us) write_op.cc:260] PREPARE: finished
0709 13:05:14.143672 (+ 19013us) write_op.cc:270] Start()
0709 13:05:14.174268 (+ 30596us) write_op.cc:276] Timestamp: P: 1783602314135715 usec, L: 0
0709 13:05:14.187039 (+ 12771us) op_driver.cc:348] REPLICATION: starting
0709 13:05:14.312242 (+125203us) log.cc:844] Serialized 7843 byte log entry
0709 13:05:15.216667 (+904425us) op_driver.cc:464] REPLICATION: finished
0709 13:05:16.090673 (+874006us) write_op.cc:301] APPLY: starting
0709 13:05:16.103814 (+ 13141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:17.168384 (+1064570us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:17.168517 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:17.319256 (+150739us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:17.333408 (+ 14152us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:05:17.333485 (+    77us) write_op.cc:312] APPLY: finished
0709 13:05:17.334234 (+   749us) log.cc:844] Serialized 172 byte log entry
0709 13:05:17.646265 (+312031us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:17.683275 (+ 37010us) write_op.cc:454] Released schema lock
0709 13:05:17.683356 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:17.695314 (+ 11958us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:17.695352 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":704656,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":765827,"apply.run_cpu_time_us":54254,"apply.run_wall_time_us":1660266,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":217458,"prepare.run_cpu_time_us":121962,"prepare.run_wall_time_us":773415,"raft.queue_time_us":398331,"raft.run_cpu_time_us":65437,"raft.run_wall_time_us":636588,"replication_time_us":969732,"thread_start_us":1280068,"threads_started":5,"wal-append.queue_time_us":773344,"wal-append.run_cpu_time_us":28037,"wal-append.run_wall_time_us":2492163}]]}
I20260709 13:05:19.860621 14983 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:19.824536 21766 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:19.653218 21194 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.716714 21505 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.143s
I20260709 13:05:19.912226 15082 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:19.928923 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.833544 21624 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.930256 14983 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 11 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:19.774870 21281 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.824154 21597 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:05:19.845909 21762 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.823992 21612 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:19.808446 21715 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:19.922102 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:19.931125 21624 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.930325 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:19.913928 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 311898us
I20260709 13:05:19.716166 21763 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.829299 21303 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:05:19.840891 21655 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.873759 21613 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:19.824282 21042 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.800628 21439 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.716274 21557 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.864073 21516 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:19.995482 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 266343us
W20260709 13:05:20.002111 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.029199 21765 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:19.941994 21725 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:19.859301 21617 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.035154 15083 raft_consensus.cc:2410] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:05:20.028996 21786 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:20.028879 21782 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:19.774308 21680 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.040940 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:19.858726 21750 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.148s
I20260709 13:05:19.842038 21756 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.167s
I20260709 13:05:19.912392 21664 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.905911 21450 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:19.959499 21612 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:19.992689 21762 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:20.125149 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:20.165139 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:20.166509 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.857283 21620 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.167232 15080 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:20.168277 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:19.927983 21766 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:20.158412 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 117644us
I20260709 13:05:20.175107 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:20.034394 21772 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:20.149363 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:20.169103 15080 raft_consensus.cc:2376] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15. Re-sending same reply.
I20260709 13:05:20.176968 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:20.177722 15080 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:20.178723 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:20.179852 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:20.150341 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:20.118381 21782 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:20.130455 21516 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:20.131930 21619 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.185773 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:20.188167 14983 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 11, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 11 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:20.163945 21612 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.164382 21771 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.205188 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.203195 15082 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:20.164585 21753 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:20.182806 21772 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.126210 21786 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:20.040342 21614 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.186636 21782 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:20.212244 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 374463us
I20260709 13:05:20.223483 15082 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:20.186020 21795 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:20.218580 14983 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 12 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.191212 21783 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:20.127507 21655 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.243942 14983 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 12, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:20.201783 21733 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.403s
I20260709 13:05:20.202436 21619 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.172097 21613 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:05:20.319799 15084 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:20.325541 21780 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:05:20.326423 21655 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.326841 21731 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.131s
I20260709 13:05:20.328621 21619 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:20.329034 21785 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:05:20.329732 21624 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.215452 21772 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:20.330574 21784 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:05:20.330669 21450 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.306638 21795 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:20.335230 21758 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.335773 21758 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.233064 21749 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.221171 21597 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.333815 21795 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.227195 21786 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:20.353220 21793 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:05:20.353535 21772 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:20.258209 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 46965.1us
I20260709 13:05:20.337697 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1543) took 1293 ms. Trace:
I20260709 13:05:20.357115 15161 rpcz_store.cc:276] 0709 13:05:19.044390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.407730 (+363340us) service_pool.cc:224] Handling call
0709 13:05:19.408079 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.408328 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.408359 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.408819 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.668497 (+259678us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:19.668530 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.668599 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.332643 (+664044us) raft_consensus.cc:1685] finished
0709 13:05:20.337525 (+  4882us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.337644 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":259280,"threads_started":1,"wal-append.queue_time_us":448286}
I20260709 13:05:20.238646 21303 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:20.311654 21488 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.953s	user 0.011s	sys 0.000s
I20260709 13:05:20.237536 21794 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:20.337495 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3094) took 1155 ms. Trace:
I20260709 13:05:20.248935 21783 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:20.238566 21452 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:20.248268 21705 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.376405 15085 rpcz_store.cc:276] 0709 13:05:19.182515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.249423 (+ 66908us) service_pool.cc:224] Handling call
0709 13:05:19.249736 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.250078 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:19.250107 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.266004 (+ 15897us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.529960 (+263956us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:19.530025 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.530135 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.337275 (+807140us) raft_consensus.cc:1685] finished
0709 13:05:20.337300 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.337440 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127804,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":248302,"threads_started":1,"wal-append.queue_time_us":584121}
I20260709 13:05:20.347378 21597 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.348069 21749 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.419582 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:20.231346 21663 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:20.349910 21779 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
I20260709 13:05:20.335294 21799 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:20.349463 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:20.351145 21795 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:20.357915 21543 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.825s	user 0.011s	sys 0.000s
W20260709 13:05:20.342612 15201 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.597s (SENT) [suppressed 26 similar messages]
W20260709 13:05:20.360736 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 185678us
I20260709 13:05:20.349510 21758 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.376603 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:20.209695 21771 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.343895 21774 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:20.375528 21303 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:20.471998 15813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.003s
I20260709 13:05:20.454663 15085 raft_consensus.cc:2410] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
W20260709 13:05:20.412019 21646 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.835s	user 0.007s	sys 0.003s
I20260709 13:05:20.380777 21619 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.375922 21794 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:20.458886 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.003s	sys 0.007s
W20260709 13:05:20.492237 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.003s	sys 0.007s
I20260709 13:05:20.267364 14983 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 10 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:20.494407 21788 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 13:05:20.494226 21619 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.493954 14983 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 8 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 11 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:20.495924 21803 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:05:20.496500 21619 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.461163 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:20.221416 21505 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:20.509198 21505 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:20.286293 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.472501 21758 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:20.462774 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:20.463655 15846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.003s
W20260709 13:05:20.519979 15846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.007s	sys 0.003s
I20260709 13:05:20.466015 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:20.522230 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56288.4us
I20260709 13:05:20.523463 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:20.408181 21783 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:20.466874 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.005s	sys 0.007s
I20260709 13:05:20.467036 15235 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 13:05:20.467562 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 91053.4us
W20260709 13:05:20.473295 15813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.010s	sys 0.003s
I20260709 13:05:20.490322 21756 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.491389 21772 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:20.464823 21634 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.001s
I20260709 13:05:20.381675 21488 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.459642 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:20.545635 15159 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:20.463362 21543 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:20.546572 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.497426 21794 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.548156 21794 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:20.498883 15162 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:20.495884 14983 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 12 to 14, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 14 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:20.552372 21505 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:20.551775 14983 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 12, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 12 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:20.501840 21807 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:20.506171 21690 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:20.506732 21597 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.508065 21620 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:20.217779 21753 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.490079 21042 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.475907 21303 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:20.521176 21791 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:20.526723 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:20.528445 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.005s	sys 0.007s
W20260709 13:05:20.531093 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 606220us
I20260709 13:05:20.476696 15084 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:20.626358 15847 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:20.535506 21795 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:20.651300 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 124668us
W20260709 13:05:20.655711 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 802076us
I20260709 13:05:20.572710 21763 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.658800 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:20.661005 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:20.460664 21799 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.626961 15156 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:20.652388 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:20.663359 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:20.629660 21775 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:05:20.493456 21664 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:20.778301 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 115049us
I20260709 13:05:20.672894 21795 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:20.723124 21588 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:20.564059 21690 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:20.804430 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:20.800083 21751 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.810638 21588 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:20.544261 21733 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.241s
I20260709 13:05:20.776391 21705 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:20.824697 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:20.564798 21766 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:20.672454 21772 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:20.800362 21597 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:20.822022 21588 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:20.783883 21620 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:20.826860 21194 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.860409 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:20.554922 21756 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.833036 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.867971 15843 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:20.869266 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:20.869863 15843 raft_consensus.cc:2376] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18. Re-sending same reply.
I20260709 13:05:20.870666 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:20.834729 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.835754 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:20.872793 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.839123 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:20.839960 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 497427us
I20260709 13:05:20.875360 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2436) took 2233 ms. Trace:
I20260709 13:05:20.877664 15159 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:20.731204 14983 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 12, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:20.853153 15840 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:05:20.876034 15086 rpcz_store.cc:276] 0709 13:05:18.630522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.630685 (+   163us) service_pool.cc:224] Handling call
0709 13:05:18.661416 (+ 30731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.113503 (+452087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.113543 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.193411 (+ 79868us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.688195 (+494784us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:19.688249 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.688333 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.863534 (+1175201us) raft_consensus.cc:1685] finished
0709 13:05:20.863559 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.863722 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53833,"raft.queue_time_us":155427,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":35639,"thread_start_us":910119,"threads_started":2,"wal-append.queue_time_us":1198174}
I20260709 13:05:20.607676 21807 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:20.886854 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.890806 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.891517 15842 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:20.889081 14983 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 8 to 10, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:20.864815 21439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.021s
I20260709 13:05:20.808650 21786 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.893725 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:20.554071 21634 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:20.878222 21796 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:20.895385 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:20.892967 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 238103us
I20260709 13:05:20.899905 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.854841 21705 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:20.900928 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:20.864570 21194 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:20.918002 15086 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:20.863618 21815 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:20.902680 21439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.011s	sys 0.021s
I20260709 13:05:20.818928 21751 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:20.895433 21749 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:20.918673 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2445) took 1089 ms. Trace:
I20260709 13:05:20.853783 21597 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.894971 21774 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:20.916136 21516 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
W20260709 13:05:21.028362 21612 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.819s	user 0.011s	sys 0.000s
I20260709 13:05:21.019089 21787 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:21.028795 21612 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.029454 21787 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:20.935401 15129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.181s	sys 0.247s
I20260709 13:05:20.934787 15845 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:21.022656 15086 rpcz_store.cc:276] 0709 13:05:19.829246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.890083 (+1060837us) service_pool.cc:224] Handling call
0709 13:05:20.911140 (+ 21057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.918408 (+  7268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.918603 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:21.032478 15129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.184s	sys 0.266s
I20260709 13:05:21.032639 15085 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:20.911545 21638 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.339s	user 0.011s	sys 0.002s
I20260709 13:05:21.031520 21612 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:20.849269 21795 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.053123 14983 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:21.077060 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2450) took 1243 ms. Trace:
I20260709 13:05:21.076350 14983 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:21.077607 15086 rpcz_store.cc:276] 0709 13:05:19.833556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.063122 (+1229566us) service_pool.cc:224] Handling call
0709 13:05:21.076411 (+ 13289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.076626 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.076663 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:21.076716 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:21.076761 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.076830 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.076993 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:20.921175 21194 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.934105 21815 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.027674 21516 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:20.967243 21771 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.092569 14983 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 9 to 10, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 10 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:21.098489 21042 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.105912 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:21.106189 21042 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.107038 21705 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.108999 21488 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.076249 21807 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.110059 21807 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:21.113793 21799 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.115298 21634 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.118546 21640 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:21.124442 21782 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.876s	user 0.002s	sys 0.005s
I20260709 13:05:21.086369 21543 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:21.127867 21794 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.001s
I20260709 13:05:20.893170 21690 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.129643 21821 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:05:21.130316 21786 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:21.131436 21812 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.311s	user 0.000s	sys 0.000s
W20260709 13:05:21.131564 21657 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.790s	user 0.013s	sys 0.001s
I20260709 13:05:21.132017 21657 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.132854 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2369) took 1459 ms. Trace:
I20260709 13:05:21.096815 21815 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:21.133229 15233 rpcz_store.cc:276] 0709 13:05:19.673092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.673259 (+   167us) service_pool.cc:224] Handling call
0709 13:05:19.673647 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:19.719439 (+ 45792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.719473 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:19.802256 (+ 82783us) log.cc:844] Serialized 52 byte log entry
0709 13:05:20.492790 (+690534us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:20.893023 (+400233us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.893271 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.893356 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.132590 (+239234us) raft_consensus.cc:1685] finished
0709 13:05:21.132609 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.132776 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45326,"prepare.queue_time_us":616519,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":55663,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":44146,"thread_start_us":999695,"threads_started":2,"wal-append.queue_time_us":781214}
W20260709 13:05:21.135982 21816 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 13:05:21.137825 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1524) took 1715 ms. Trace:
I20260709 13:05:21.138163 15846 rpcz_store.cc:276] 0709 13:05:19.422674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.424335 (+  1661us) service_pool.cc:224] Handling call
0709 13:05:19.455383 (+ 31048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.689420 (+234037us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:19.798981 (+109561us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.520591 (+721610us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.521183 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.521303 (+   120us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:20.521332 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.521391 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.137615 (+616224us) raft_consensus.cc:1685] finished
0709 13:05:21.137640 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.137769 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":109703,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":103843,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":3542,"thread_start_us":922121,"threads_started":2,"wal-append.queue_time_us":764690}
I20260709 13:05:21.140414 21597 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.100349 21641 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:21.024919 21753 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.148459 21649 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:21.150739 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1527) took 1613 ms. Trace:
I20260709 13:05:21.071784 21801 raft_consensus.cc:2764] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:05:21.150571 14983 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 2 to 13, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 13 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:21.153299 15847 rpcz_store.cc:276] 0709 13:05:19.537680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.537867 (+   187us) service_pool.cc:224] Handling call
0709 13:05:19.559096 (+ 21229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.650941 (+1091845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.150491 (+499550us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400580054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.150679 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454008192}
I20260709 13:05:21.084197 21439 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.056995 21787 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.157510 15155 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:21.053227 21789 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.495s
I20260709 13:05:21.165742 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36620 (request call id 4) took 1259 ms. Trace:
I20260709 13:05:21.169512 14983 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 12, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 12 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:21.166220 15086 rpcz_store.cc:276] 0709 13:05:19.906123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.081524 (+1175401us) service_pool.cc:224] Handling call
0709 13:05:21.165639 (+ 84115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:21.177800 15841 raft_consensus.cc:2368] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16.
I20260709 13:05:21.179576 15847 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:05:21.157343 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 292026us
I20260709 13:05:21.141108 21042 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.084758 21505 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.160140 15083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1570) took 1697 ms (client timeout 1760 ms). Trace:
W20260709 13:05:21.191205 15083 rpcz_store.cc:269] 0709 13:05:19.462606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.586654 (+124048us) service_pool.cc:224] Handling call
0709 13:05:21.159940 (+1573286us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540025ba54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.160083 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720650496}
W20260709 13:05:21.184674 21783 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.001s
I20260709 13:05:21.166175 21782 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.186650 21516 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.187071 21281 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.167630 21439 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.188483 21690 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:21.199735 15083 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:21.261008 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1578) took 1223 ms. Trace:
I20260709 13:05:21.261533 15083 rpcz_store.cc:276] 0709 13:05:20.037843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.192365 (+1154522us) service_pool.cc:224] Handling call
0709 13:05:21.199419 (+  7054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.260651 (+ 61232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.260926 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:21.160027 21641 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.260658 21516 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:21.342695 21303 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.565s	user 0.000s	sys 0.009s
I20260709 13:05:21.343183 21303 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.345768 21825 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:05:21.199820 21505 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.200376 21783 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.348726 21439 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.364943 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1546) took 1527 ms. Trace:
I20260709 13:05:21.349783 21628 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:21.365554 15156 rpcz_store.cc:276] 0709 13:05:19.837740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.838943 (+  1203us) service_pool.cc:224] Handling call
0709 13:05:19.839202 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.673785 (+834583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.364579 (+690794us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.364856 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811021184}
I20260709 13:05:21.383248 21628 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.383841 21789 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.384397 21789 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.385593 21628 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.386179 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:21.386816 15229 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 12.
I20260709 13:05:21.386869 21789 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.387938 15233 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:05:21.388190 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3093) took 2206 ms. Trace:
I20260709 13:05:21.389209 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:21.388725 15088 rpcz_store.cc:276] 0709 13:05:19.182253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.242405 (+ 60152us) service_pool.cc:224] Handling call
0709 13:05:19.242671 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.603851 (+361180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.387901 (+1784050us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54003cf354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.388115 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603247872}
W20260709 13:05:21.396059 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:21.340421 21516 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.407397 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:21.401274 21439 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:05:21.422173 21797 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:21.422693 21797 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:21.423568 21797 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.425741 21819 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:21.428290 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2377) took 1081 ms. Trace:
I20260709 13:05:21.428723 15235 rpcz_store.cc:276] 0709 13:05:20.347218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.349889 (+  2671us) service_pool.cc:224] Handling call
0709 13:05:20.350206 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.528497 (+178291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.427968 (+899471us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003d95d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.428230 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267917696}
I20260709 13:05:21.430778 21752 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:21.431200 21752 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:21.441504 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.444123 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:21.444731 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:21.443542 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:21.454298 15156 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:21.456036 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:21.439815 21740 consensus_meta.cc:287] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.563s	user 0.011s	sys 0.002s
I20260709 13:05:21.459106 21603 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:21.459649 21829 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:21.461033 21750 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:21.461663 21641 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.462255 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:21.468442 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:21.402360 21751 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.441150 21797 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:21.472777 21762 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.289s	user 0.008s	sys 0.004s
I20260709 13:05:21.432406 21752 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.497150 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.503670 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:21.463402 21657 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.521179 15232 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 13.
I20260709 13:05:21.521008 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:21.478324 21740 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.536774 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:21.537139 15154 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:21.505571 21762 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.515100 21782 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.524652 21752 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.529456 21750 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:21.548225 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1574) took 1363 ms. Trace:
W20260709 13:05:21.549908 15813 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.686s (SENT)
I20260709 13:05:21.483491 21829 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.554656 15080 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:21.548796 15845 rpcz_store.cc:276] 0709 13:05:20.186532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.186681 (+   149us) service_pool.cc:224] Handling call
0709 13:05:20.192094 (+  5413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.034683 (+842589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.547886 (+513203us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b54006f0254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.548130 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1707330304}
I20260709 13:05:21.533054 21303 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.538506 21590 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:21.551529 21750 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.582455 21829 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.567584 21783 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.584034 21703 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.592562 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:21.594220 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.595743 15156 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:21.582489 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3077) took 2619 ms. Trace:
W20260709 13:05:21.584347 21588 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.000s
I20260709 13:05:21.559808 21590 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:21.600474 15154 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:21.601945 21741 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.603224 21657 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:21.598619 15163 rpcz_store.cc:276] 0709 13:05:18.962610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.371279 (+408669us) service_pool.cc:224] Handling call
0709 13:05:19.371622 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.822867 (+451245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.822913 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.824895 (+  1982us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.129698 (+304803us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:20.129753 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.581964 (+1452211us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54002878d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.582034 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.582097 (+    63us) raft_consensus.cc:1685] finished
0709 13:05:21.582115 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.582404 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2401,"raft.queue_time_us":16935,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7010,"spinlock_wait_cycles":313015168,"thread_start_us":514676,"threads_started":2,"wal-append.queue_time_us":369120,"wal-append.run_cpu_time_us":2458,"wal-append.run_wall_time_us":1261637}
I20260709 13:05:21.569988 21750 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.570159 21516 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.602140 21641 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:05:21.594424 21799 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.730674 21840 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:05:21.730868 21590 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.732337 21839 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:05:21.626474 21588 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.645198 21703 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.737352 15230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2361) took 3003 ms (client timeout 3071 ms). Trace:
W20260709 13:05:21.742321 15230 rpcz_store.cc:269] 0709 13:05:18.722966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.723077 (+   111us) service_pool.cc:224] Handling call
0709 13:05:21.726134 (+3003057us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54000582d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.726328 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796209024}
I20260709 13:05:21.737828 15843 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
W20260709 13:05:21.732920 21834 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:05:21.599677 21815 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.729434 21505 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.737711 21741 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.735252 21646 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:21.740869 21588 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.755718 21762 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.772308 21731 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.738600 21799 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.772744 21795 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:21.732811 21590 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:21.803321 21795 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.806604 21838 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:21.806846 21795 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.734045 21740 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.814494 21815 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:21.809170 21731 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.833448 21557 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.835958 21543 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:21.836575 21543 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:21.808970 21756 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:21.837714 21543 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:21.839497 21543 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:21.841540 21690 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.001s
I20260709 13:05:21.793712 21646 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:21.846130 21646 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.847580 21799 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.847841 21646 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:21.848675 21787 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:21.849314 21787 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:21.850490 21787 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.852427 21787 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:21.843137 21831 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
W20260709 13:05:21.827996 21845 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:05:21.754462 21703 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.860149 21703 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:21.854468 21731 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:21.920109 15080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2448) took 2089 ms (client timeout 2560 ms). Trace:
I20260709 13:05:21.955668 21281 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.955629 15080 rpcz_store.cc:269] 0709 13:05:19.830067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.179604 (+349537us) service_pool.cc:224] Handling call
0709 13:05:21.919402 (+1739798us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.919765 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3160106752}
W20260709 13:05:21.921880 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 309457us
I20260709 13:05:21.957437 21266 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.959304 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.958752 21266 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.961542 21487 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:21.963112 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1611) took 1125 ms. Trace:
I20260709 13:05:21.962916 21487 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.963743 15843 rpcz_store.cc:276] 0709 13:05:20.837182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.870436 (+ 33254us) service_pool.cc:224] Handling call
0709 13:05:21.962583 (+1092147us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54004b55d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.962876 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942820096}
I20260709 13:05:21.922488 15087 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:05:21.965041 15087 pending_rounds.cc:85] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 7
I20260709 13:05:21.965468 15087 pending_rounds.cc:107] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:05:21.965885 15087 raft_consensus.cc:2889] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.966004 21634 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.967317 21634 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.935341 21042 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.798409 15842 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 13. Current term is 14.
W20260709 13:05:21.941501 21194 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.002s
W20260709 13:05:21.944113 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.952255 15082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2447) took 2122 ms (client timeout 2195 ms). Trace:
W20260709 13:05:21.954478 15084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2446) took 2124 ms (client timeout 1651 ms). Trace:
W20260709 13:05:21.978621 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.972316 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3096) took 2771 ms. Trace:
I20260709 13:05:21.964641 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2644) took 2303 ms. Trace:
I20260709 13:05:21.980918 15087 rpcz_store.cc:276] 0709 13:05:19.197073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.293898 (+ 96825us) service_pool.cc:224] Handling call
0709 13:05:19.355576 (+ 61678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.966636 (+2611060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.968356 (+  1720us) spinlock_profiling.cc:243] Waited 2.57 s on lock 0x7b5400349254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:21.968657 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5900470016}
I20260709 13:05:21.971750 21042 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:21.975648 15084 rpcz_store.cc:269] 0709 13:05:19.829553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.867507 (+ 37954us) service_pool.cc:224] Handling call
0709 13:05:21.953886 (+2086379us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54000db6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.954171 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655343872}
W20260709 13:05:21.997563 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.975517 15082 rpcz_store.cc:269] 0709 13:05:19.829819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.930070 (+100251us) service_pool.cc:224] Handling call
0709 13:05:21.946200 (+2016130us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b5400307354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:21.952076 (+  5876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194678144}
I20260709 13:05:22.001551 21439 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:05:22.013238 21799 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:22.014204 21799 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.017756 21799 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.018282 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3095) took 2835 ms. Trace:
I20260709 13:05:22.018757 15079 rpcz_store.cc:276] 0709 13:05:19.182783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.259451 (+ 76668us) service_pool.cc:224] Handling call
0709 13:05:19.259900 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:19.724342 (+464442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.724487 (+   145us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:19.725842 (+  1355us) log.cc:844] Serialized 52 byte log entry
0709 13:05:19.948430 (+222588us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:20.033855 (+ 85425us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.915820 (+1881965us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b1800254680. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3c8f9a40 00007f0d3c8f44e4 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287 00007f0d3f9ff208
0709 13:05:21.988365 (+ 72545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.991660 (+  3295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.991728 (+    68us) raft_consensus.cc:1685] finished
0709 13:05:21.999658 (+  7930us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.018120 (+ 18462us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40760,"prepare.queue_time_us":90860,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":2038405,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":456,"spinlock_wait_cycles":158696832,"thread_start_us":482380,"threads_started":2,"wal-append.queue_time_us":382287,"wal-append.run_cpu_time_us":2350,"wal-append.run_wall_time_us":1149285}
I20260709 13:05:21.876495 21750 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:21.980967 15234 rpcz_store.cc:276] 0709 13:05:19.660989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.661475 (+   486us) service_pool.cc:224] Handling call
0709 13:05:19.728066 (+ 66591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.728569 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:19.804425 (+ 75856us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.755910 (+1951485us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b18004e01c0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a
0709 13:05:21.756078 (+   168us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.756606 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.756856 (+   250us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:21.963043 (+206187us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.963754 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.963898 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.964069 (+   171us) raft_consensus.cc:1685] finished
0709 13:05:21.964089 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.964433 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":229298,"prepare.queue_time_us":283646,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"raft.queue_time_us":102171,"raft.run_cpu_time_us":834,"raft.run_wall_time_us":23840,"spinlock_wait_cycles":14396800,"thread_start_us":104332,"threads_started":1,"wal-append.queue_time_us":536033}
I20260709 13:05:22.064975 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1544) took 3007 ms. Trace:
I20260709 13:05:22.065707 15155 rpcz_store.cc:276] 0709 13:05:19.044683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.432252 (+387569us) service_pool.cc:224] Handling call
0709 13:05:19.432589 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.164932 (+1732343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.050232 (+885300us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54003e1f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:22.052235 (+  2003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3966544128}
I20260709 13:05:22.067926 21487 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.074291 21634 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:22.076826 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.046455 21281 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.025309 21194 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.079603 21281 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.076668 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3101) took 2353 ms. Trace:
I20260709 13:05:22.090636 15158 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:22.104076 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.097559 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.102970 15843 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:22.081430 15846 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:22.107342 15159 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: [12.6-14.7]
I20260709 13:05:22.056885 21752 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.091483 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1617) took 1196 ms. Trace:
I20260709 13:05:22.095394 21750 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:22.101176 21705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.049s	sys 0.113s
I20260709 13:05:22.136787 15842 rpcz_store.cc:276] 0709 13:05:20.895051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.895171 (+   120us) service_pool.cc:224] Handling call
0709 13:05:22.091124 (+1195953us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54005a3a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:22.091311 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045309824}
I20260709 13:05:22.125123 15085 rpcz_store.cc:276] 0709 13:05:19.723145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.499631 (+776486us) service_pool.cc:224] Handling call
0709 13:05:20.500053 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.064254 (+564201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.075961 (+1011707us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:22.076389 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224412544}
I20260709 13:05:22.080190 21194 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.098968 21042 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:22.151566 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.104538 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.153437 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:22.161216 21851 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:22.162582 21851 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.095301 21797 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:22.165427 21797 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.165427 21753 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.167282 21753 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.165715 21852 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:22.095638 21632 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:22.156661 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.101552 21711 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:22.089432 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.103786 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:22.152873 21705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.052s	sys 0.113s
I20260709 13:05:22.080962 21765 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:22.181872 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:22.182868 21765 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.183710 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.185567 21852 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:22.187364 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 44139.6us
W20260709 13:05:22.188963 15201 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.822s (SENT)
I20260709 13:05:22.197861 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.168286 21797 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.203286 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:22.199174 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.193354 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.199743 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:22.207650 15839 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.013s	sys 0.033s
I20260709 13:05:22.212092 15155 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
W20260709 13:05:22.219765 15839 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.014s	sys 0.034s
I20260709 13:05:22.223378 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.224400 15157 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:22.205991 21852 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.190202 21042 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:22.224483 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1621) took 1070 ms. Trace:
I20260709 13:05:22.230132 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.230937 21690 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:22.231284 15843 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:22.231931 21690 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.203732 21853 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:22.233335 21853 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.235572 21690 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.235754 21853 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.236539 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.237571 15087 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:22.237334 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.242717 15847 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:22.243893 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.244975 21632 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.246115 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.247582 21632 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.248219 15085 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:05:22.250820 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.252189 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.252951 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.253309 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.217729 21194 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:05:22.254776 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.255350 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.255596 21194 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:22.256013 21194 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:22.256155 15236 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.254458 15234 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:22.244560 15847 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:22.259119 15085 pending_rounds.cc:85] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 13
I20260709 13:05:22.226836 15846 rpcz_store.cc:276] 0709 13:05:21.154079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.154270 (+   191us) service_pool.cc:224] Handling call
0709 13:05:21.999405 (+845135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.130223 (+130818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.224114 (+ 93891us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54004b55d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:22.224386 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856368512}
I20260709 13:05:22.232759 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.229035 15840 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:22.269238 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.272217 15085 pending_rounds.cc:107] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:05:22.272631 15085 raft_consensus.cc:2889] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:22.273406 15162 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:22.273825 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.198136 21752 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.278839 21851 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.279734 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:22.276602 15201 leader_election.cc:336] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.894s (SENT)
W20260709 13:05:22.281571 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61702.8us
W20260709 13:05:22.282253 21855 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.100s	user 0.004s	sys 0.000s
I20260709 13:05:22.284011 21543 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:22.284744 21848 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:05:22.285645 21833 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:22.286777 21439 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:22.287263 21439 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:22.288671 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.289600 15156 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:22.289216 21042 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.290190 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:22.290695 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.291220 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.292114 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1618) took 1387 ms. Trace:
I20260709 13:05:22.292573 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.292503 15839 rpcz_store.cc:276] 0709 13:05:20.904374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.904491 (+   117us) service_pool.cc:224] Handling call
0709 13:05:20.920312 (+ 15821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.160192 (+239880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.160220 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.164309 (+  4089us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.379534 (+215225us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:22.260218 (+880684us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.291592 (+ 31374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.291741 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.291808 (+    67us) raft_consensus.cc:1685] finished
0709 13:05:22.291819 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.292052 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98878,"prepare.queue_time_us":712118,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":106134,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":33400,"thread_start_us":1170749,"threads_started":3,"wal-append.queue_time_us":187720}
I20260709 13:05:22.293499 15157 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:22.295718 15839 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:05:22.297175 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.297927 15082 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:22.299226 21753 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.300045 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.300863 15158 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:22.300931 15084 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:22.300169 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.288515 21439 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.304306 15232 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:22.302100 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.305477 21439 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:22.302240 15156 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:22.280702 15842 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:22.303275 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.307624 15087 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:22.200259 21711 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.274914 15237 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:22.310563 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.315145 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.322708 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.311450 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.326766 15236 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:22.325580 15161 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:22.331979 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:22.332944 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.335045 21752 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:22.336125 21266 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.336623 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:22.339041 21711 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.341241 15847 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:22.343140 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.343884 15237 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.276974 21853 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:22.362648 21853 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.350099 21194 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.364102 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1575) took 2223 ms. Trace:
I20260709 13:05:22.364645 21194 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:22.366767 21590 consensus_meta.cc:287] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.000s
I20260709 13:05:22.371152 21853 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:22.374226 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 157862us
I20260709 13:05:22.372952 15840 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:22.373055 21042 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.373453 21797 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:22.374531 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.375173 15841 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:22.375939 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.396389 15237 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
W20260709 13:05:22.377136 21646 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.528s	user 0.015s	sys 0.009s
I20260709 13:05:22.377164 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.378616 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:22.379235 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89248us
I20260709 13:05:22.386468 21711 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:22.393065 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:22.350204 21858 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:05:22.396073 15234 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:22.407475 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.364537 15086 rpcz_store.cc:276] 0709 13:05:20.036871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.173096 (+1136225us) service_pool.cc:224] Handling call
0709 13:05:21.173462 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.202390 (+1028928us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:22.206257 (+  3867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.206309 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:22.206352 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.215921 (+  9569us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.259810 (+ 43889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":750882,"raft.queue_time_us":791434,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":324,"thread_start_us":203421,"threads_started":1}
I20260709 13:05:22.411058 15162 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:22.404888 15235 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:22.414777 15844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1608) took 1578 ms. Trace:
I20260709 13:05:22.414880 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.415660 15233 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 11.
I20260709 13:05:22.415269 15844 rpcz_store.cc:276] 0709 13:05:20.836394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.836561 (+   167us) service_pool.cc:224] Handling call
0709 13:05:20.837122 (+   561us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:20.837756 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.837799 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:20.838774 (+   975us) log.cc:844] Serialized 78 byte log entry
0709 13:05:22.014090 (+1175316us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b18004d3bc0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:22.293474 (+279384us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:22.301203 (+  7729us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.301525 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.301673 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.414258 (+112585us) raft_consensus.cc:1685] finished
0709 13:05:22.414374 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.414591 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22264,"prepare.queue_time_us":4815,"raft.queue_time_us":12085,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":12301,"spinlock_wait_cycles":86210560,"thread_start_us":154144,"threads_started":1,"wal-append.queue_time_us":1434732}
I20260709 13:05:22.419052 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.419983 15231 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:22.425884 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.437634 15233 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:05:22.439361 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.441149 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:22.441687 15201 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.935s (SENT)
I20260709 13:05:22.442446 15163 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:22.444059 15201 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.011s (SENT)
I20260709 13:05:22.445013 21799 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:22.453438 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.459408 15844 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:22.461104 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.461400 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.461905 15844 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:22.462873 15842 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:22.464152 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.471107 15087 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 23.
I20260709 13:05:22.459318 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.485181 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.429221 21646 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:22.494525 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.499907 15848 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:22.501662 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.477474 21042 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:05:22.530822 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.534515 15843 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:05:22.535982 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 92011.9us
I20260709 13:05:22.538082 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.540180 15234 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:22.549787 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.541290 21690 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:22.556584 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:22.560144 15840 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:22.557682 15163 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:22.564499 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:22.555896 21690 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.575421 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:22.573544 15842 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:22.585872 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:22.546561 21646 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.566026 21281 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:22.586031 21690 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.595739 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.598364 15840 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 13: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:22.603188 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:22.603084 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 223265us
I20260709 13:05:22.594738 21543 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:22.610126 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 63304.4us
W20260709 13:05:22.611441 15201 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.100s (SENT)
I20260709 13:05:22.613227 21765 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.614110 21765 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.615072 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1579) took 2576 ms. Trace:
I20260709 13:05:22.615480 15083 rpcz_store.cc:276] 0709 13:05:20.038144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.262485 (+1224341us) service_pool.cc:224] Handling call
0709 13:05:21.337269 (+ 74784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.155246 (+817977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.155310 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.158103 (+  2793us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.266962 (+108859us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:22.543164 (+276202us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.553108 (+  9944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.562209 (+  9101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.614677 (+ 52468us) raft_consensus.cc:1685] finished
0709 13:05:22.614699 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.614880 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":601938,"prepare.queue_time_us":337156,"raft.queue_time_us":635994,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":4700,"thread_start_us":483397,"threads_started":3,"wal-append.queue_time_us":386467}
I20260709 13:05:22.609709 21543 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.620586 15080 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:22.622071 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.625309 15082 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:22.626899 15843 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:22.622319 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.636430 15085 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:22.621553 15087 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:22.646114 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.646786 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.643117 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.649071 15840 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:22.617471 21705 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:22.650432 21705 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:22.651268 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52280us
W20260709 13:05:22.652179 15201 leader_election.cc:336] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.121s (SENT)
I20260709 13:05:22.651820 21705 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.653497 21705 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:22.617681 21634 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.656155 21634 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.659106 21634 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.660776 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.662002 15080 raft_consensus.cc:2376] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17. Re-sending same reply.
I20260709 13:05:22.662415 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.663323 15081 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:22.664374 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.665051 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.665769 15079 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:22.675915 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.616801 21765 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.684754 21194 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:22.685336 21194 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:22.697037 21860 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:05:22.697191 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.698217 15840 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:22.685797 21194 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:22.639472 21641 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:22.701354 21641 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:22.619957 21543 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:22.702524 21641 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.704382 21641 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:22.676950 21850 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.368s	user 0.002s	sys 0.001s
I20260709 13:05:22.678175 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.679054 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.710491 15847 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:22.710765 21646 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.679538 21799 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:22.711423 21799 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.711890 21799 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:22.711752 21731 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:22.616973 21557 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:22.680387 15839 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:22.715744 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.713871 21557 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.681185 21281 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:22.725734 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49929us
I20260709 13:05:22.652710 15201 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:22.762117 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 91642.2us
I20260709 13:05:22.762715 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.763502 15081 raft_consensus.cc:2376] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19. Re-sending same reply.
I20260709 13:05:22.764506 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.765199 15081 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:22.766028 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.766574 15081 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:22.767369 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.767901 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:22.768947 21711 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.769711 21711 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.772389 21711 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:22.768759 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.780884 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.782188 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.783305 15081 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:22.789374 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:22.790330 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.801568 21634 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.802695 21638 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.802496 21634 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.803416 21638 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.805491 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.805511 21634 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.806107 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.806488 21634 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:22.685662 21868 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:22.807446 21868 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.808481 21266 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:22.809080 21266 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:22.810199 21868 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:22.811088 21439 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.004s
I20260709 13:05:22.811736 21439 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:22.813959 21646 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:22.814610 21646 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.816479 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.813958 21638 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.817495 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.818219 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.818126 21266 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.818733 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.819363 15080 raft_consensus.cc:2376] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17. Re-sending same reply.
I20260709 13:05:22.820637 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.820767 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.821064 21646 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:22.757006 21557 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:22.822363 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.825640 15088 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:22.756480 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:22.827152 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.760758 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:22.828760 15082 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:22.812297 21439 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.835069 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.830953 15843 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:22.830085 15080 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:22.832744 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 69845.2us
I20260709 13:05:22.819834 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.843128 15085 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
W20260709 13:05:22.844524 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 115955us
I20260709 13:05:22.849382 15086 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:22.851903 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.848011 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.861001 15088 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:22.863511 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:22.864007 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.865913 15085 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:22.842120 21711 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:22.863312 15081 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:22.864846 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.895264 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.901793 15085 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:05:22.861078 21752 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:22.811101 21634 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.914974 15081 raft_consensus.cc:1275] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:05:22.917014 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:22.918282 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.915524 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.921361 15087 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:22.921895 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:22.893975 21646 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:22.921891 15079 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:22.925770 15847 raft_consensus.cc:2410] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:05:22.926138 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:22.913579 15051 connection.cc:256] RPC call timeout handler was delayed by -1.3125468190000902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 13:05:22.918627 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 266523us
I20260709 13:05:22.928565 15088 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [12.10-14.12]
W20260709 13:05:22.921787 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 408921us
I20260709 13:05:22.920007 21752 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:22.935698 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.936153 15082 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:22.936738 15201 leader_election.cc:336] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.163s (SENT)
I20260709 13:05:22.937875 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.938081 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.940623 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:22.938215 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.942283 15081 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:22.944272 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:22.900897 21487 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:05:22.946442 15237 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:22.947733 15083 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:22.949852 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:22.950752 15087 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:22.917738 21853 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:05:22.953771 15230 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:22.957384 21439 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:22.958009 21439 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:22.958328 21439 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.958631 21439 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.959445 21543 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:22.962464 21752 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.943490 21042 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:05:22.966938 21042 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:22.967489 21042 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:22.969352 21854 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:22.969794 21854 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:22.970857 21854 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.974366 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:22.975742 15237 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:22.977104 15237 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:22.977417 15233 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:22.978246 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.978895 15237 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:22.974709 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.981675 15236 raft_consensus.cc:2368] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:22.975521 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:22.973914 21557 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.983476 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:22.984649 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 120700us
I20260709 13:05:22.985389 21557 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.986002 15230 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:22.986440 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:22.987824 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:22.988901 21634 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:22.990280 21833 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:22.991127 21439 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:23.001698 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71994.8us
I20260709 13:05:22.995056 15237 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:22.968641 21042 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:23.004266 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72990.2us
W20260709 13:05:23.005097 15201 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.220s (SENT)
I20260709 13:05:23.005587 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:23.006042 21863 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:23.007874 15232 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:23.009514 21731 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:23.009807 15087 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:05:23.010241 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57582.1us
I20260709 13:05:23.010846 15082 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:05:23.017067 15079 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:22.949102 21638 raft_consensus.cc:2764] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:23.023226 21799 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.027113 21799 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.028035 21799 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:23.021595 21557 raft_consensus.cc:2764] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:05:23.029043 21868 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:22.991689 21833 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.019980 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:23.024291 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:22.972681 21854 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.031018 15232 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:23.035449 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.039057 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:23.018079 21705 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:23.053822 21797 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:23.041817 15231 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.029s	sys 0.052s
W20260709 13:05:23.055640 15231 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.029s	sys 0.052s
I20260709 13:05:22.952492 21646 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:22.992159 21711 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:23.066013 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:23.069798 15085 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:23.051463 21638 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.084025 21801 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.096194 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.111379 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:23.041884 21690 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.113060 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:23.117035 15126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.014s	sys 0.017s
W20260709 13:05:23.120190 15126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.014s	sys 0.018s
I20260709 13:05:23.096570 21638 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:23.121791 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:23.121440 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 545905us
I20260709 13:05:23.126406 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:23.127434 15082 raft_consensus.cc:2368] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:23.120653 21853 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.120108 21543 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:23.123586 21638 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:23.142572 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.138260 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:23.138939 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.145517 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.108974 21801 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.120391 21705 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 13:05:23.212363 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.228645 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:23.229189 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.229928 15079 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:05:23.230355 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 98546.7us
I20260709 13:05:23.141029 21853 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.231176 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.131580 21634 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:05:23.155813 21632 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.246104 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.226235 21753 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:23.268702 15079 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:23.234686 21853 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:23.245497 21194 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:05:23.275295 21194 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.279858 15839 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:23.282215 15843 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:23.249408 21439 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.248936 21543 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.266921 21641 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:23.281890 21867 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:23.304916 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.297443 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 66332.9us
W20260709 13:05:23.306661 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.310382 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.317087 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.327579 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.294889 21690 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.311338 21867 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.357983 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 552598us
W20260709 13:05:23.318222 21850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.025s	sys 0.068s
I20260709 13:05:23.366868 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.295213 21632 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.368983 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.373733 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.375787 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:23.376609 15085 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:23.372037 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 438684us
W20260709 13:05:23.372049 21850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.026s	sys 0.068s
I20260709 13:05:23.352353 21439 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.360586 21867 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:23.385975 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.387524 15163 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:23.381208 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 353220us
I20260709 13:05:23.359853 21641 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:23.371845 21543 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:23.372256 21869 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:23.357614 21851 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:05:23.420229 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 103245us
W20260709 13:05:23.416762 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 427589us
I20260709 13:05:23.422402 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.419188 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:23.427048 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68128.1us
I20260709 13:05:23.431257 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.431509 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.387650 21874 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:23.432503 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:23.393134 21543 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:23.414762 21690 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:23.413800 21869 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:23.389292 21439 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:23.446359 15845 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:23.435907 15160 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:23.448405 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:23.449352 15088 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.450498 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:23.484615 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45995.7us
I20260709 13:05:23.442589 21543 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:23.484064 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 462051us
I20260709 13:05:23.437321 21874 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:23.488516 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 57353.4us
I20260709 13:05:23.495498 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:23.441970 21690 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:23.503458 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 65395.2us
I20260709 13:05:23.496344 15839 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:23.502549 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 397193us
I20260709 13:05:23.522017 21042 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.538537 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.552117 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.552556 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.552973 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.553328 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:23.592237 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:23.592662 15162 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:23.553206 21641 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.554541 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.601749 15087 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:23.602298 15847 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:23.554452 21869 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:23.555403 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:23.521339 21875 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:05:23.486152 21557 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:23.555640 21487 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:23.556470 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.498924 21194 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:23.546887 21870 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.418s	user 0.005s	sys 0.000s
W20260709 13:05:23.536962 15810 connection.cc:256] RPC call timeout handler was delayed by -2.150839732999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:05:23.500895 21801 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:23.538023 15201 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.597s (SENT)
I20260709 13:05:23.558184 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.558233 21646 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:23.559760 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.563467 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.621773 15157 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:23.564131 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:23.537249 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 105803us
I20260709 13:05:23.566437 21705 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:23.522339 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.579030 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:23.653461 15844 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.575909 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:23.579538 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:23.623538 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:23.478519 21873 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:23.661774 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:23.498164 21872 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 13:05:23.591733 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 446651us
I20260709 13:05:23.600029 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:23.577965 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:23.665153 15846 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:23.586490 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.487849 21752 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:23.537791 21871 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 13:05:23.606446 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:23.497685 21874 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.616693 15842 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:05:23.487188 21883 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.523689 21690 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:23.628114 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:23.631091 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92657.5us
I20260709 13:05:23.633986 21439 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:23.637017 21281 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.611943 21731 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:05:23.615216 21042 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:23.645674 15163 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:23.693207 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 106805us
I20260709 13:05:23.645370 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:23.694231 15229 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.619835 21853 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.646468 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:23.695027 15233 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:23.647394 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:23.647728 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.649590 15160 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:23.654739 15845 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:23.615424 21487 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:23.662869 15159 raft_consensus.cc:2393] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:23.662812 15085 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:05:23.663166 15232 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.621s	user 0.002s	sys 0.010s
I20260709 13:05:23.624555 21632 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:23.663744 15088 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:23.664635 15841 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.614784 21557 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:23.668111 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.668020 21799 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:23.668517 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:23.667999 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:23.672513 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 375461us
I20260709 13:05:23.699833 15232 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:23.672127 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.670689 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:23.677543 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.700522 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:23.649605 21869 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.679399 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:23.673265 21891 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.120s	user 0.004s	sys 0.000s
W20260709 13:05:23.683202 15810 connection.cc:256] RPC call timeout handler was delayed by -1.0256049480000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:05:23.685772 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73259.8us
I20260709 13:05:23.662093 21873 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:23.685074 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2393) took 1306 ms. Trace:
I20260709 13:05:23.685551 21753 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:23.677909 21705 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:23.673440 15839 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:23.699770 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47767.1us
I20260709 13:05:23.679407 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.682153 21194 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.702145 21557 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:23.712922 15154 raft_consensus.cc:2410] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:05:23.716492 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.714517 15054 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.719468 15082 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:23.719846 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.703100 15161 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:23.726974 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1452) took 1309 ms. Trace:
I20260709 13:05:23.690090 21752 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:23.699267 21634 raft_consensus.cc:2764] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:05:23.731815 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 193857us
I20260709 13:05:23.708212 15228 rpcz_store.cc:276] 0709 13:05:22.378503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.396505 (+ 18002us) service_pool.cc:224] Handling call
0709 13:05:22.439489 (+ 42984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.440101 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.440150 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.440873 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.842851 (+401978us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:22.842895 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.842972 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.546085 (+703113us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:23.682476 (+136391us) raft_consensus.cc:1685] finished
0709 13:05:23.682498 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.684970 (+  2472us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":245249,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":3833216,"thread_start_us":387815,"threads_started":1,"wal-append.queue_time_us":643639}
I20260709 13:05:23.702737 21799 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.760828 15155 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:23.738862 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:23.739020 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 66407.7us
I20260709 13:05:23.739868 21646 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:23.736222 21194 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.722607 21868 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:23.741817 21873 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.742982 21487 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:23.744262 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96653.8us
I20260709 13:05:23.745008 15845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "12daa67457954a0cab27e32bf85fa115"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:23.735946 15156 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:23.736258 21266 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:23.749266 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:23.734098 21883 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:23.749758 21881 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 13:05:23.767316 21799 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.750309 21879 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:23.752612 21797 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:23.753513 21892 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:05:23.754535 21638 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.002s
I20260709 13:05:23.755993 21042 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:23.756420 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.759534 15810 leader_election.cc:400] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:23.759917 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.702581 21899 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:23.735782 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.727653 15235 rpcz_store.cc:276] 0709 13:05:22.417771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.420937 (+  3166us) service_pool.cc:224] Handling call
0709 13:05:22.421570 (+   633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.422077 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:22.422608 (+   531us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.948808 (+526200us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.965314 (+ 16506us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.965467 (+   153us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:22.965707 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.010330 (+ 44623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.010410 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.726705 (+716295us) raft_consensus.cc:1685] finished
0709 13:05:23.726736 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.726898 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54903,"prepare.queue_time_us":7776,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":113984,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":23651,"thread_start_us":401216,"threads_started":1,"wal-append.queue_time_us":660806}
I20260709 13:05:23.767848 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.773660 15839 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:23.773804 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.790436 21731 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:23.733776 21854 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.697s	user 0.002s	sys 0.010s
I20260709 13:05:23.778316 15846 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:05:23.779353 15201 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.657s (SENT)
I20260709 13:05:23.805420 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.749472 21753 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.784225 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.771231 21752 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.712049 21281 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:05:23.786954 15154 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:23.787452 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 359297us
I20260709 13:05:23.816176 15844 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:23.781669 15845 raft_consensus.cc:606] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Received request to transfer leadership to TS 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:23.750296 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.776914 21868 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.796363 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57605.8us
I20260709 13:05:23.799793 15159 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:05:23.800771 15810 leader_election.cc:403] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:05:23.784291 21883 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.792475 21880 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 13:05:23.741024 21853 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.783861 21266 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:23.806375 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 69706.6us
I20260709 13:05:23.786082 21900 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:23.760802 21705 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.817950 15079 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:23.849207 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.803782 21487 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.830202 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.823149 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.804759 21889 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.790957 21801 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:23.803694 21851 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:23.824055 21868 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:23.844660 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 109600us
I20260709 13:05:23.846982 21557 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.803123 21879 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:23.805361 21731 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.816936 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.837184 21266 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:23.884555 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67713.6us
W20260709 13:05:23.870514 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 111196us
I20260709 13:05:23.852263 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.843616 21905 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:23.875536 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 100346us
I20260709 13:05:23.802170 21638 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:23.877310 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47801.2us
I20260709 13:05:23.848783 21854 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:23.886461 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.891752 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.837342 21883 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:23.892874 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 156641us
I20260709 13:05:23.893306 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:23.895567 15128 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.853211 21281 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.876315 21042 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:23.891402 21731 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:23.920693 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 90619.6us
I20260709 13:05:23.893245 21641 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:23.925356 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:23.908797 21646 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.894819 21854 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:23.932790 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45037us
I20260709 13:05:23.933497 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:23.926810 21439 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:23.932075 21632 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:23.936671 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 139673us
I20260709 13:05:23.919723 21851 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.937929 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.941425 21853 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:23.941831 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.927285 21042 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:23.945252 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:23.929026 21797 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.935640 21905 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.936394 21439 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:23.963174 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.937999 21641 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:23.965211 21711 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:23.966003 21914 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:23.893297 21889 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.968003 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:23.972783 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.974813 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:23.975818 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:23.976243 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:23.976390 15839 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.968717 21878 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:23.946645 21281 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:23.986200 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 79377.9us
I20260709 13:05:23.943799 21904 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:23.936420 21901 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.187s
I20260709 13:05:23.967073 21905 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:23.994580 15845 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:23.966447 21194 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:23.990412 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47473us
I20260709 13:05:23.969535 21851 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.969007 21439 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.992806 21919 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:23.992974 21876 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:23.971707 21914 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.943819 21632 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:23.976914 21711 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:23.999485 15844 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:24.018028 15841 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:24.007920 21913 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:24.008359 21903 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:24.009016 21902 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:05:24.010512 21908 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 13:05:24.012398 21557 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:23.987931 21690 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.013778 21876 consensus_queue.cc:1243] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:24.047679 21690 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.007202 21854 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:24.049826 21925 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:24.050272 21854 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.998095 21905 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:24.054176 21851 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:24.054785 21851 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.058104 21889 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.059316 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:24.059803 21799 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:24.060873 21910 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:05:24.061136 15161 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:24.061950 21711 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.063484 21873 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.029565 15229 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:24.010699 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.065068 21752 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:24.030375 15127 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:23.977105 21797 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.042053 15839 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:24.042822 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.043670 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:24.121711 15847 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:05:24.045298 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:24.110667 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 128345us
W20260709 13:05:24.120435 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90150.6us
I20260709 13:05:24.125823 21281 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:24.126922 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:24.127779 15841 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:24.127298 21281 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.128851 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:24.131707 21894 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 13:05:24.136310 21543 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.001s
I20260709 13:05:24.136682 21915 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:05:24.136978 21641 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.139348 21797 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:24.140362 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:24.146427 21920 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:05:24.149446 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 204305us
W20260709 13:05:24.150890 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 187816us
I20260709 13:05:24.151897 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:24.153784 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 209539us
I20260709 13:05:24.063450 21711 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:24.157119 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:24.036916 21904 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:24.038087 21913 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:24.178560 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 171414us
W20260709 13:05:24.189528 15128 connection.cc:256] RPC call timeout handler was delayed by -1.0987286010000616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
I20260709 13:05:24.190692 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:24.149192 21889 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:24.029487 21914 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.232568 15154 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:24.246938 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:24.248013 15841 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:24.248705 15810 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:24.155766 21851 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.144284 21439 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.260531 15845 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:24.122715 21900 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.111s
W20260709 13:05:24.263602 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 70106.8us
I20260709 13:05:24.263820 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:23.991725 21879 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.185737 21904 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:24.195590 21881 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:24.136783 21543 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.280536 15839 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:24.242878 21638 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:24.289216 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 120152us
W20260709 13:05:24.275736 21910 log.cc:927] Time spent T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:05:24.293097 21929 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.282167 21878 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:24.294349 21898 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:24.295512 21934 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.300511 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:24.195829 21913 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:24.172156 21042 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:24.305757 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 135576us
I20260709 13:05:24.272671 21851 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:24.379617 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 79230us
I20260709 13:05:24.306169 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2395) took 1900 ms. Trace:
I20260709 13:05:24.287601 21904 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.317199 21690 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.198104 21801 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:05:24.330253 15158 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:24.287447 21881 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.374709 21945 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:24.374868 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:24.375782 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 178809us
I20260709 13:05:24.375886 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:24.418054 15082 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:24.376338 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:24.377283 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:24.307504 21929 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.286819 21557 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.414585 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 223981us
W20260709 13:05:24.420469 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49013.7us
I20260709 13:05:24.425352 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:24.427062 15128 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:24.380206 15231 rpcz_store.cc:276] 0709 13:05:22.424302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.424504 (+   202us) service_pool.cc:224] Handling call
0709 13:05:22.479521 (+ 55017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.133914 (+654393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.133958 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:23.225427 (+ 91469us) log.cc:844] Serialized 26 byte log entry
0709 13:05:23.661774 (+436347us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:23.661809 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.661877 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.305871 (+643994us) raft_consensus.cc:1685] finished
0709 13:05:24.305890 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.306015 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4624,"raft.queue_time_us":209285,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":174,"thread_start_us":897542,"threads_started":2,"wal-append.queue_time_us":513007}
W20260709 13:05:24.430465 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 255512us
I20260709 13:05:24.429448 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:24.432431 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:24.432979 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:24.433338 15083 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:05:24.434208 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:24.436604 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:24.417297 21881 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:24.434373 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:24.439437 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:24.442273 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:24.379385 21638 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:05:24.438642 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:24.444090 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:24.455415 15810 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:24.461440 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 263457us
I20260709 13:05:24.444465 21752 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:24.293823 21922 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:05:24.296226 21903 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:24.444413 21799 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:24.408480 21934 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.428862 21557 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.469375 21851 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:05:24.502218 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62872.1us
I20260709 13:05:24.468068 21752 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:24.510349 15154 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:05:24.515830 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.454340 21945 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.521678 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:24.524611 15082 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:24.526600 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.510462 21939 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:24.511467 21752 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.532928 15839 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:24.519023 21867 consensus_meta.cc:287] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.093s	user 0.011s	sys 0.002s
W20260709 13:05:24.518723 21869 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.812s	user 0.011s	sys 0.000s
I20260709 13:05:24.540230 21867 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:24.543671 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 73159.7us
I20260709 13:05:24.471287 21905 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.518955 21705 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.552726 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:24.533550 21919 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:24.510345 21799 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:05:24.550997 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60960.3us
I20260709 13:05:24.546482 15084 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:24.558951 15841 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:24.533655 21926 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:24.560360 21931 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.560895 21928 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:24.545001 21938 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:24.552677 21937 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:24.566025 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.569276 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.570087 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.571099 21853 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:24.558506 21927 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:24.573457 21933 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:05:24.573624 21945 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:24.658330 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.659049 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2662) took 1096 ms. Trace:
I20260709 13:05:24.659231 15232 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:24.574908 21869 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.528738 21936 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:24.659569 15160 rpcz_store.cc:276] 0709 13:05:23.562131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.791341 (+229210us) service_pool.cc:224] Handling call
0709 13:05:23.799110 (+  7769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.803758 (+  4648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.803800 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:23.820043 (+ 16243us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.153019 (+332976us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:24.153072 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.153170 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.658823 (+505653us) raft_consensus.cc:1685] finished
0709 13:05:24.658846 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.658995 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":273075,"threads_started":1,"wal-append.queue_time_us":419951}
I20260709 13:05:24.561816 21942 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:05:24.565136 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:24.630295 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 185042us
I20260709 13:05:24.544432 21932 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:24.555300 21543 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.549329 21934 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:24.647172 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 100747us
W20260709 13:05:24.648537 15201 leader_election.cc:336] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.986s (SENT)
I20260709 13:05:24.648614 21962 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:24.629873 21938 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:24.650470 21867 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:24.652339 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:24.655805 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:24.732136 15083 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.509689 21878 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.639082 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.670711 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.673905 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:24.574429 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.740413 15231 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
W20260709 13:05:24.643996 21487 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.762s	user 0.011s	sys 0.000s
I20260709 13:05:24.556288 21705 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:24.689288 21731 consensus_meta.cc:287] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.000s
I20260709 13:05:24.690330 21914 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.691293 21950 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:24.691606 21557 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.724167 21889 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:24.694075 21646 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.001s
I20260709 13:05:24.638067 21853 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:24.699905 21874 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.012s	user 0.008s	sys 0.002s
I20260709 13:05:24.706657 21632 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:24.709506 21929 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:24.710003 21930 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:05:24.649894 21905 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:24.789232 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:24.678969 21266 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.773s	user 0.007s	sys 0.004s
I20260709 13:05:24.724187 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:24.659502 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.740782 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.684737 21869 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.729523 21833 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:24.746160 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 93932.5us
W20260709 13:05:24.729816 21854 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 13:05:24.684032 21936 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:24.737229 21939 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:24.683280 21928 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:24.738313 21868 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.834s	user 0.005s	sys 0.006s
I20260709 13:05:24.716320 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:24.829439 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 143580us
I20260709 13:05:24.759887 21731 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.773540 15230 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:24.773217 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.772466 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:24.763424 21950 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:24.768234 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:24.803310 15229 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:24.771646 21851 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:24.790128 15236 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
W20260709 13:05:24.835266 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 193123us
I20260709 13:05:24.740079 21962 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.850443 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:24.800910 21976 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:24.842367 15231 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:24.798079 21932 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:24.841984 21950 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.856937 15234 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:24.793152 21874 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.862776 15228 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:24.863256 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.835871 21646 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:24.866519 21970 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:24.867187 21705 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:24.745985 21913 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.836229 21936 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:24.886371 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1567) took 1539 ms. Trace:
W20260709 13:05:24.870299 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 221790us
I20260709 13:05:24.873924 21853 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.874356 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:24.879845 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.885393 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:24.888471 15840 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:24.886806 15158 rpcz_store.cc:276] 0709 13:05:23.330264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.353226 (+ 22962us) service_pool.cc:224] Handling call
0709 13:05:23.369716 (+ 16490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.409420 (+1039704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.869578 (+460158us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540017e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:24.869767 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208424320}
I20260709 13:05:24.893751 15236 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:24.798710 21266 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.895460 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.896942 15237 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:24.902734 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:24.909528 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 146193us
W20260709 13:05:24.888211 21873 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.001s
I20260709 13:05:24.842590 21868 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.848384 21939 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.915633 15233 raft_consensus.cc:2368] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:24.924808 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:24.921769 15201 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.229s (SENT)
I20260709 13:05:24.961720 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:24.915721 15813 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:24.998978 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 74351.8us
I20260709 13:05:24.881282 21833 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:24.896442 21851 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:25.034318 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:25.044412 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:25.007311 21904 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.004s
I20260709 13:05:25.000324 15128 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:24.896601 21042 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:24.970501 21853 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:24.968720 21557 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.057713 21557 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.837394 21938 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.051596 15235 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:25.058897 21833 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.061621 21557 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:24.861088 21928 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.065244 21869 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:25.066429 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 168204us
I20260709 13:05:25.068804 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:25.069517 21752 consensus_meta.cc:287] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.006s
I20260709 13:05:25.069620 15158 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:25.070421 21752 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.071203 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:25.072021 21966 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:25.071810 21904 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.072438 21966 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:25.072701 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 157066us
I20260709 13:05:25.072962 21966 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.074159 15813 leader_election.cc:400] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:25.074333 21904 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.075162 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.076239 21874 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.076862 21914 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.078023 15088 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:25.079123 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78913.4us
I20260709 13:05:25.080542 21986 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.070704 21854 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.083130 15128 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:25.082973 21854 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.084172 21873 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.084863 21873 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.085470 21854 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.086804 21851 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.087975 15847 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:25.089249 21976 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:25.089048 21942 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:05:25.090207 15160 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:05:25.090764 21942 consensus_queue.cc:1048] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:25.092128 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.054389 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:24.921471 21970 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:25.096737 15201 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.670s (SENT)
I20260709 13:05:25.097383 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:25.087366 21873 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.093851 21942 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:25.038328 21895 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:25.072515 21868 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.070449 21876 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.105538 21869 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.110041 21976 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:25.072544 21752 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.133471 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.114933 21986 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:25.115957 21883 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.187s	user 0.006s	sys 0.007s
I20260709 13:05:25.116369 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:25.111955 21904 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.117597 21942 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:25.117815 15201 leader_election.cc:336] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.961s (SENT)
I20260709 13:05:25.118180 21903 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:25.118394 21914 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:25.113345 15810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.007s
W20260709 13:05:25.142637 15810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.000s	sys 0.008s
I20260709 13:05:25.136920 21869 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.143038 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 589076us
I20260709 13:05:25.143715 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:25.144544 15236 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:25.144687 21869 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.132925 21976 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.113759 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:25.145442 21976 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.150861 21266 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:25.153110 21968 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:05:25.153126 21731 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:25.154701 21690 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.000s
I20260709 13:05:25.155135 21690 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:25.157415 21913 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.157574 21942 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:24.917476 21905 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 13:05:25.140686 21903 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:25.161702 21903 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.162879 21903 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.053066 21879 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:25.163946 21879 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:25.164047 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:25.108111 21711 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.929s	user 0.007s	sys 0.005s
I20260709 13:05:25.165344 15229 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:25.164529 21879 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.116740 21966 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:24.904942 21889 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.167598 21966 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.167936 21797 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:25.168403 21797 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:25.053805 21932 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.169361 21797 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.170616 21797 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.171587 21974 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.172122 21974 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.128305 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.139958 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:25.173873 21962 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:25.128070 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.122187 21854 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:25.165165 21970 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.121181 21439 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.137686 15231 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:25.139127 21931 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:25.185709 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 69508.7us
I20260709 13:05:25.188932 15083 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:25.151889 15230 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:25.205761 21927 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:25.207334 21972 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.321s	user 0.002s	sys 0.000s
W20260709 13:05:25.208143 21937 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.000s
I20260709 13:05:25.190636 21895 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:25.208890 21895 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:25.142360 21904 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.209336 21895 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.211033 21904 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.209353 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.212982 15236 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:25.208647 21937 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:25.218519 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.218538 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.221233 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.225751 15845 raft_consensus.cc:2368] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
W20260709 13:05:25.226989 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.227963 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.229453 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.230144 15234 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:25.231109 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.231899 15234 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:25.233081 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.234103 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:25.234897 15236 raft_consensus.cc:2410] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:25.236819 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.242946 15846 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:25.237211 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.241024 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.238615 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:25.252279 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55520.4us
I20260709 13:05:25.240793 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.252971 15844 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:05:25.237308 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 119512us
I20260709 13:05:25.253657 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:25.140882 21883 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.271370 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.255156 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.262182 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:25.283987 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.278031 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:25.267472 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.245895 15163 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:25.285980 15841 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.265720 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.284969 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:25.292613 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:25.291558 15054 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:25.297266 15842 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:25.298241 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.299381 15844 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.300391 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.301194 15844 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:25.301929 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.302057 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.302753 15844 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:25.303059 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.303761 15846 raft_consensus.cc:2368] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:05:25.271613 21913 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.305238 21913 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.305992 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.306396 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.307092 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.307576 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3126) took 1498 ms. Trace:
I20260709 13:05:25.308655 15161 raft_consensus.cc:2368] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20.
I20260709 13:05:25.309747 21883 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.308151 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.307936 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:25.300716 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46517.3us
I20260709 13:05:25.311163 15082 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:25.303592 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.261654 21879 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.312301 15844 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:25.304528 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.314672 15846 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
W20260709 13:05:25.239773 21881 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.753s	user 0.003s	sys 0.007s
I20260709 13:05:25.317725 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.318789 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.319304 15839 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:25.307929 15081 rpcz_store.cc:276] 0709 13:05:23.808940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.809111 (+   171us) service_pool.cc:224] Handling call
0709 13:05:23.809523 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.012477 (+202954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.012509 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:24.012555 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:24.012605 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.307370 (+1294765us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540034ab54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:25.307415 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.307528 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452989696}
I20260709 13:05:25.321796 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.325891 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.237278 21962 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:25.319245 15843 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:25.313928 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:25.316174 21939 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:25.259739 21974 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.329002 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.329893 15841 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:25.329617 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.225054 21927 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:25.330470 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.330703 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.331082 15842 raft_consensus.cc:2376] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16. Re-sending same reply.
I20260709 13:05:25.331498 15841 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.330937 21927 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.332439 15842 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:05:25.332985 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.336020 15845 raft_consensus.cc:2368] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
W20260709 13:05:25.337213 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47377.4us
I20260709 13:05:25.338151 15845 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:25.340962 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48488us
I20260709 13:05:25.342164 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.242763 21936 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.329165 21939 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:25.332805 15842 pending_rounds.cc:85] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 10
I20260709 13:05:25.351037 15842 pending_rounds.cc:107] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:05:25.290923 21928 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:25.351516 15842 raft_consensus.cc:2889] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:25.351706 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.346887 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.316638 21929 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:25.310405 21753 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:25.357751 15156 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.358467 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:25.359298 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.360635 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:25.363196 15079 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:05:25.363739 15232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.015s	sys 0.017s
W20260709 13:05:25.364171 15232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.015s	sys 0.017s
I20260709 13:05:25.345192 21914 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:25.373965 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.374387 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:25.374970 15842 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 13.
I20260709 13:05:25.345561 21989 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.364430 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:25.376978 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.354403 21042 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:25.379055 15845 raft_consensus.cc:2393] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:25.375754 15839 raft_consensus.cc:2410] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:25.356956 21931 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.258358 21950 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.358140 21993 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:25.395589 21994 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.378819 21042 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.396683 21042 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.396710 21950 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.397095 21879 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.325865 21869 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:05:25.397737 15160 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.397434 21989 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.398283 21931 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.399669 21989 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.377866 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.240646 21976 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.392366 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.400458 21931 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.363360 21883 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:25.395648 15158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457"
dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34534
I20260709 13:05:25.410162 15158 raft_consensus.cc:606] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.410271 21869 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:25.410758 21869 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:25.411711 21869 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.362228 21753 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:25.413080 21753 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.413686 21869 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.400777 15842 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.401469 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.417364 15844 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:25.393076 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:25.402595 21938 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:25.329057 21881 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.405429 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.396080 21994 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:25.405963 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.407868 21281 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:25.377667 15201 leader_election.cc:400] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:05:25.418452 15844 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:25.416620 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:25.402091 21976 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.415469 21963 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:25.409022 21936 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.435545 21895 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.425557 15084 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:25.426748 15848 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:25.428388 15231 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:25.376785 21948 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:25.455473 15842 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:25.373920 21914 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:25.464931 21543 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:25.465092 21914 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.465409 21543 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:25.466496 21543 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.467588 21543 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:25.470461 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.471607 21833 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.472143 21833 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.473639 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:25.475593 21895 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:25.410203 21969 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.357s	user 0.002s	sys 0.000s
W20260709 13:05:25.434633 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.458s (SENT) [suppressed 46 similar messages]
W20260709 13:05:25.477577 21977 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.398s	user 0.003s	sys 0.000s
I20260709 13:05:25.478161 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.478322 21881 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.479806 21973 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:25.479873 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:25.480274 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.435123 21930 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:25.480886 21958 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:25.481173 21930 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.482290 21958 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:25.482649 21958 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.482992 21958 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:25.483198 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.483474 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.438308 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.484623 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.428453 21994 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.485445 15084 raft_consensus.cc:2410] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:05:25.486016 15230 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:25.486217 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.486805 15084 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:25.486980 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.487351 15230 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:05:25.425359 21281 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:25.487833 15085 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:25.488302 21281 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.441090 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.442481 21939 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.445371 15234 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:25.448511 15844 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:05:25.427662 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.490020 15839 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:25.373059 21929 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:25.484268 15081 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:05:25.459329 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.492630 15154 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.460009 15840 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:25.493269 15840 raft_consensus.cc:606] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.493726 15840 raft_consensus.cc:614] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:25.461231 21909 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:25.495316 15842 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:05:25.463224 21979 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.392s	user 0.000s	sys 0.002s
I20260709 13:05:25.463567 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:25.472263 21042 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.489485 15230 pending_rounds.cc:85] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 6
I20260709 13:05:25.438429 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.501523 15230 pending_rounds.cc:107] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:05:25.495203 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.496363 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:25.426329 15845 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 14: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:25.496912 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.499853 15236 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:25.437866 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60325.5us
I20260709 13:05:25.501791 15230 raft_consensus.cc:2889] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:25.418371 15232 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.527031 15233 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:25.506556 21903 raft_consensus.cc:2764] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:25.535550 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.501124 21909 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:25.534842 15839 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:25.537501 15158 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.540001 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.538461 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:25.543260 15236 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:25.545248 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:25.546015 15084 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:25.531781 21281 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:25.554610 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51368us
I20260709 13:05:25.535626 21711 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.555814 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.541301 21883 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:25.557960 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.559023 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.563988 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.556667 15201 leader_election.cc:400] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:05:25.551139 21915 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:25.631156 15232 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:25.633041 15154 raft_consensus.cc:2393] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.608547 15081 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.608067 15084 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.637585 15237 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:25.632598 15201 leader_election.cc:403] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:05:25.650635 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 94942.9us
W20260709 13:05:25.649564 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 93061.3us
I20260709 13:05:25.656078 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.657163 15083 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.537144 21929 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.563476 21711 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:25.639271 21833 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.650354 21962 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13.
I20260709 13:05:25.668413 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.609416 21883 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.691362 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.695080 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:25.697032 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:25.700786 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.702718 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:25.704819 15083 raft_consensus.cc:2393] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:25.635603 21909 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.643080 21915 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:25.707090 21999 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:05:25.739850 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.701751 21895 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.756047 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.735760 21915 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:25.759716 15157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2653) took 2195 ms (client timeout 2053 ms). Trace:
I20260709 13:05:25.761389 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:25.760135 15157 rpcz_store.cc:269] 0709 13:05:23.559709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.674619 (+114910us) service_pool.cc:224] Handling call
0709 13:05:25.749427 (+2074808us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400086bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:25.755119 (+  5692us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3088657664}
I20260709 13:05:25.738803 21983 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:25.770712 15839 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:25.759016 21909 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:25.773105 21042 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.775460 21883 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:25.758169 21895 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.779426 21989 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:25.765270 21976 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:25.770804 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.785614 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.786474 15839 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:25.781648 21976 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.790510 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.792810 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:25.796144 15842 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:25.811833 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.816183 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.817167 15154 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.803421 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:25.819293 15845 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:25.779206 21895 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:25.790413 21973 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:25.847615 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.781674 21989 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:05:25.866446 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.780966 21983 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.869099 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 352973us
I20260709 13:05:25.849764 21851 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.889482 15840 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:25.911971 15839 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:25.895618 22002 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:05:25.848387 21973 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.924726 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:25.913897 21711 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:25.938666 21973 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:25.933935 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.943336 15051 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.883s (SENT)
I20260709 13:05:25.945477 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:25.899042 21851 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:25.934237 21878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.028s	sys 0.076s
W20260709 13:05:25.935037 22003 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:05:25.948504 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:25.950546 15157 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:25.944038 21881 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:25.954950 21973 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:25.972512 15051 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.913s (SENT)
I20260709 13:05:25.973101 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:25.982698 21543 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.003s
W20260709 13:05:25.954234 21878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.029s	sys 0.076s
I20260709 13:05:25.967100 21881 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:25.988902 21976 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:25.997869 21936 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:26.004467 21938 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.003s
I20260709 13:05:26.011432 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:26.036415 15081 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:26.009769 21994 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.020s
I20260709 13:05:26.022181 15084 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
W20260709 13:05:26.012568 21963 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.573s	user 0.024s	sys 0.004s
W20260709 13:05:26.028614 22004 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:05:26.048530 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.047338 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.012629 21543 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.081079 21543 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.083401 21543 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.040185 21973 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:26.084249 21973 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.086334 21973 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.087154 15810 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:26.087500 21994 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:26.088279 21938 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.088724 21976 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:26.088794 21938 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.089627 21976 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:26.091210 21938 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.090875 21976 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.091264 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:26.053095 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:26.054986 15163 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:26.059186 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.059707 21981 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:26.060482 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.061226 21984 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.992s	user 0.000s	sys 0.003s
W20260709 13:05:26.062045 22008 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:05:26.062323 21963 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:26.062232 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:26.107918 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45780.9us
I20260709 13:05:26.108609 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:26.109897 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:26.062942 22006 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:05:26.066789 21928 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:26.044938 21731 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.074944 21983 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.037967 21948 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.101099 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:26.061312 22009 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:05:26.050002 15084 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:26.102787 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.096390 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:26.134748 15086 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:26.107324 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.108273 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.146086 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:26.100113 21851 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:26.144284 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47965.4us
I20260709 13:05:26.115123 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.146836 15157 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.103942 15154 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:26.149674 15813 leader_election.cc:400] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:26.149948 15813 leader_election.cc:403] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:26.145565 15840 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 11.
I20260709 13:05:26.104815 21281 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.116312 21914 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:05:26.113958 21981 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.154350 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.110996 21913 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.118235 21909 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:26.120198 21989 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.118423 21928 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:26.148495 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:26.117524 21194 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:26.151000 21962 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:26.111948 21973 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:05:26.152432 21895 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:26.161841 21958 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 13:05:26.138664 21731 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.205354 15084 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:26.208457 15081 pending_rounds.cc:85] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 12
I20260709 13:05:26.209053 15081 pending_rounds.cc:107] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:26.209498 15081 raft_consensus.cc:2889] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.198021 15154 raft_consensus.cc:1217] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Deduplicated request from leader. Original: 9.7->[11.8-17.10]   Dedup: 11.8->[15.9-17.10]
I20260709 13:05:26.157361 21914 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:26.187433 21869 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260709 13:05:26.204306 15161 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:26.222311 15844 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:26.223109 15161 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:26.187134 21913 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:26.201567 15086 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:26.212126 15154 pending_rounds.cc:85] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 8
I20260709 13:05:26.234539 15154 pending_rounds.cc:107] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:26.234874 15154 raft_consensus.cc:2889] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.235281 15842 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:05:26.186688 21194 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.154393 21851 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:05:26.188110 21927 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16.
I20260709 13:05:26.185925 21962 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.217365 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:26.154784 21281 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:26.191953 15202 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:26.227221 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.151400 21948 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.228317 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:26.229615 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 80058.2us
I20260709 13:05:26.191015 21895 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:26.214190 21753 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.801s	user 0.014s	sys 0.000s
I20260709 13:05:26.240550 15843 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:26.188586 21936 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:05:26.267839 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:26.277983 15842 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
W20260709 13:05:26.269295 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52031.8us
W20260709 13:05:26.272403 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 80533.4us
W20260709 13:05:26.280364 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.280668 15054 leader_election.cc:400] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20.
W20260709 13:05:26.280803 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.270915 15839 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:26.272182 15086 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:26.273114 15844 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.273476 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:26.270299 21962 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.274653 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:26.286027 15847 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.241070 21194 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.212482 21958 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.289394 15845 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:26.293444 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.294373 15842 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:26.300648 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:26.272336 21543 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:26.303911 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.277278 21895 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.274483 21939 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:26.277411 21711 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:26.318866 15084 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: [13.12-15.13]
I20260709 13:05:26.309186 15086 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:05:26.311303 15839 raft_consensus.cc:2393] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:26.306149 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 193760us
W20260709 13:05:26.307622 21915 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.000s
I20260709 13:05:26.329514 15845 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:26.306201 21927 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:26.324716 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 424012us
I20260709 13:05:26.328575 15127 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:26.338111 15127 leader_election.cc:403] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:26.307971 21543 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.342096 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.322363 15844 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:26.323144 21889 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:05:26.310398 21909 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:26.328044 21711 raft_consensus.cc:697] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:26.322903 21881 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.333437 21915 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.307844 21903 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:26.326000 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.320526 21939 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:26.344158 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1642) took 1093 ms. Trace:
I20260709 13:05:26.288023 21753 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.341378 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:26.321563 21922 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:05:26.340354 21641 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.357198 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.366649 15052 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:26.322700 21989 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:05:26.366994 15052 leader_election.cc:403] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:26.345943 21543 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.369758 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3138) took 1510 ms. Trace:
I20260709 13:05:26.358675 15841 rpcz_store.cc:276] 0709 13:05:25.250576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.332087 (+ 81511us) service_pool.cc:224] Handling call
0709 13:05:25.332374 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.332630 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:25.765090 (+432460us) log.cc:844] Serialized 14 byte log entry
0709 13:05:26.336717 (+571627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:26.336797 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:26.343815 (+  7018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.343918 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.344088 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":589616,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":7989,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":966775,"threads_started":3,"wal-append.queue_time_us":491417}
I20260709 13:05:26.334441 21928 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.373906 15080 rpcz_store.cc:276] 0709 13:05:24.859581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.859764 (+   183us) service_pool.cc:224] Handling call
0709 13:05:24.867860 (+  8096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.905437 (+ 37577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.905479 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.956704 (+ 51225us) log.cc:844] Serialized 26 byte log entry
0709 13:05:26.104895 (+1148191us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18003d3840. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:26.306506 (+201611us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:26.306563 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.322188 (+ 15625us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.369374 (+ 47186us) raft_consensus.cc:1685] finished
0709 13:05:26.369401 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.369671 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26218,"raft.queue_time_us":42486,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":41112,"spinlock_wait_cycles":3111808,"thread_start_us":201402,"threads_started":1,"wal-append.queue_time_us":1399084}
I20260709 13:05:26.378934 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.340067 21970 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:05:26.388232 15088 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:26.373366 22007 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:05:26.360497 22019 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.358130 21981 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.403386 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:26.400341 15841 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:26.374204 21881 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:26.412097 21881 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.412606 21881 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:26.427400 22023 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:26.428130 22023 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:26.432898 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.404225 15080 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:26.434342 15086 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:05:26.378895 21913 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.431036 21928 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:26.438608 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.442061 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.434870 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:26.387531 21909 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:26.375572 22021 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:26.447522 15086 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:05:26.447259 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80738.3us
I20260709 13:05:26.448536 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.399164 21989 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:26.450989 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.434741 21970 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:26.459739 15235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2401) took 1888 ms (client timeout 1526 ms). Trace:
W20260709 13:05:26.463773 15235 rpcz_store.cc:269] 0709 13:05:24.561096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.568978 (+  7882us) service_pool.cc:224] Handling call
0709 13:05:26.449711 (+1880733us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003d9fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:26.449842 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936800000}
I20260709 13:05:26.395834 21942 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:05:26.458196 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.477062 15157 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:26.445039 22021 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.478549 15161 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:26.483294 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.484987 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.445317 21909 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:26.486946 15232 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:26.449707 21914 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:26.454634 21929 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.498536 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:26.484830 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.501770 15157 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:26.504803 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.505476 15237 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.461809 21970 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:26.509728 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.513684 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.516286 15230 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:26.518136 15229 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:26.519008 15231 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:26.445175 21913 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.510017 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2406) took 1886 ms. Trace:
I20260709 13:05:26.521787 15228 rpcz_store.cc:276] 0709 13:05:24.624123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.624270 (+   147us) service_pool.cc:224] Handling call
0709 13:05:24.675510 (+ 51240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.863371 (+187861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.509773 (+1646402us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400175f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:26.509958 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427290496}
I20260709 13:05:26.524705 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:26.524452 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.525483 15232 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:26.485637 21963 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:26.527604 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:26.489617 22027 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.529496 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:26.533560 15235 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:26.495045 21281 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:26.489665 22021 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.543231 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 181155us
I20260709 13:05:26.465569 21989 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.546006 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.546892 15230 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.541729 15839 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:26.449759 21711 consensus_queue.cc:237] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:26.545823 15233 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:26.544255 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:26.554342 15237 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:26.515281 21976 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:26.539031 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:26.519842 21958 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.499792 22031 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:26.565557 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.577196 15232 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:26.555218 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:26.568503 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:26.577942 15228 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.536926 21973 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:26.584055 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.544405 21927 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:26.693225 21927 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:26.649257 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68178.3us
I20260709 13:05:26.432010 22019 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.659551 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:26.699272 22019 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:26.700076 22018 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:05:26.700673 15232 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:26.585024 21973 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.661933 21913 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.662613 21881 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.825582 15230 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:26.665153 15237 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:05:26.665588 21889 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:26.666131 21989 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:26.660579 21948 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:26.666558 22028 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:05:26.567471 21976 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:26.570281 21878 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:26.668231 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.485914 21939 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.575270 21922 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:26.689618 22023 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:26.585662 15235 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:26.690721 21965 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.615s	user 0.006s	sys 0.008s
I20260709 13:05:26.580749 21915 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:26.702593 15839 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
W20260709 13:05:26.801966 15847 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.515s	user 0.023s	sys 0.010s
I20260709 13:05:26.863154 15847 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:26.866539 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2484) took 2458 ms. Trace:
I20260709 13:05:26.866936 15087 rpcz_store.cc:276] 0709 13:05:24.408399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.408691 (+   292us) service_pool.cc:224] Handling call
0709 13:05:24.421419 (+ 12728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.439878 (+ 18459us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:24.492560 (+ 52682us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.643410 (+150850us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.644069 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.644245 (+   176us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:24.644537 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.973790 (+329253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.238041 (+264251us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b18002d48a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:25.298983 (+ 60942us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b54003250d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:25.299060 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.410649 (+111589us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18002d48a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:26.580984 (+1170335us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:26.866335 (+285351us) raft_consensus.cc:1685] finished
0709 13:05:26.866356 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.866484 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53811,"prepare.queue_time_us":124616,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":116,"raft.queue_time_us":204998,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":7360,"spinlock_wait_cycles":142322560,"thread_start_us":493759,"threads_started":3,"wal-append.queue_time_us":582335}
I20260709 13:05:26.701556 22037 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:26.669155 21753 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:26.857410 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.846866 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:26.859289 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.860268 21878 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.860677 21976 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.546732 22027 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.557439 21909 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.795359 21973 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.873842 15228 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:26.833573 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:26.900454 15840 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:26.875082 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 426637us
W20260709 13:05:26.876509 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.886041 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.901636 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.896885 22037 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:26.898170 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.659922 21970 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:26.884299 22036 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:26.911414 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:26.888024 21878 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.902362 15157 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:26.918751 15156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2674) took 1755 ms (client timeout 1599 ms). Trace:
I20260709 13:05:26.927870 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.939895 21042 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:26.940636 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:26.941985 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:26.942607 15082 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:26.928193 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:26.945138 15080 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:26.945359 22035 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:05:26.945616 15162 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1575) took 2970 ms (client timeout 1532 ms). Trace:
I20260709 13:05:26.928098 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:26.946010 15162 rpcz_store.cc:269] 0709 13:05:23.975440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.976008 (+   568us) service_pool.cc:224] Handling call
0709 13:05:26.945412 (+2969404us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54001c5cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:26.945533 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4750646528}
I20260709 13:05:26.946738 15086 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:26.947597 15162 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:26.928982 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 494185us
I20260709 13:05:26.940490 21042 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:26.938356 15156 rpcz_store.cc:269] 0709 13:05:25.163614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.163785 (+   171us) service_pool.cc:224] Handling call
0709 13:05:26.918457 (+1754672us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540013e7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:26.918645 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435501312}
I20260709 13:05:26.948805 15162 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:26.914609 21939 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.930485 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:26.947800 21909 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.951805 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:26.958124 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:26.919464 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 279095us
I20260709 13:05:26.953759 21915 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.950163 15228 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:26.947499 21194 consensus_meta.cc:287] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.001s
I20260709 13:05:27.005786 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:27.048805 15235 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:27.013743 15232 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:27.014110 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:27.050921 15237 raft_consensus.cc:2368] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:27.027714 15157 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 19 index: 17. (term mismatch)
I20260709 13:05:27.052031 15157 pending_rounds.cc:85] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 16
I20260709 13:05:27.052305 15157 pending_rounds.cc:107] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 18.17
I20260709 13:05:27.052574 15157 raft_consensus.cc:2889] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.035043 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1618) took 1631 ms. Trace:
I20260709 13:05:27.054191 15085 rpcz_store.cc:276] 0709 13:05:25.403999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.488463 (+ 84464us) service_pool.cc:224] Handling call
0709 13:05:26.957929 (+1469466us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800003698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449aa0a0 00007f0d449a9984 00007f0d3df246d4 00007f0d3df24662
0709 13:05:27.034980 (+ 77051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23905152}
I20260709 13:05:27.039424 15845 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:05:27.039747 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:27.034150 15230 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:27.060027 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.043941 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:27.033604 15156 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:26.915275 22036 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:27.061806 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 215042us
W20260709 13:05:27.066982 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.295811 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 225261us
I20260709 13:05:27.297417 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:27.014729 21996 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.609s	user 0.009s	sys 0.005s
W20260709 13:05:27.029860 22024 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.499s	user 0.007s	sys 0.021s
I20260709 13:05:27.305215 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:26.947067 22027 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:26.997069 22047 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.320246 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3139) took 2437 ms (client timeout 1970 ms). Trace:
I20260709 13:05:27.064389 21939 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:27.319648 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1626) took 2085 ms. Trace:
I20260709 13:05:27.438253 21939 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:27.007730 21883 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:27.438768 21939 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:27.323971 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.438474 15848 rpcz_store.cc:276] 0709 13:05:25.234042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.239879 (+  5837us) service_pool.cc:224] Handling call
0709 13:05:25.240237 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.454926 (+214689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.319351 (+1864425us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540071e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:27.319572 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407620864}
I20260709 13:05:27.442364 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:27.443706 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:27.444303 15848 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
W20260709 13:05:27.437978 15079 rpcz_store.cc:269] 0709 13:05:24.882541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.882749 (+   208us) service_pool.cc:224] Handling call
0709 13:05:27.320021 (+2437272us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400306954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.320172 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056515968}
I20260709 13:05:27.063493 22023 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.446403 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:27.446946 22023 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.447840 15848 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.449271 22023 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:27.455163 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1609) took 2112 ms. Trace:
I20260709 13:05:27.455732 15081 rpcz_store.cc:276] 0709 13:05:25.342420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.182732 (+840312us) service_pool.cc:224] Handling call
0709 13:05:26.208220 (+ 25488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:26.210346 (+  2126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:26.210400 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:26.211378 (+   978us) log.cc:844] Serialized 26 byte log entry
0709 13:05:26.354769 (+143391us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:26.929148 (+574379us) log.cc:844] Serialized 14 byte log entry
0709 13:05:26.929361 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.929669 (+   308us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:27.029243 (+ 99574us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:27.454760 (+425517us) raft_consensus.cc:1685] finished
0709 13:05:27.454895 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:27.455083 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68381,"prepare.queue_time_us":144670,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":3676,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17941,"spinlock_wait_cycles":7781120,"thread_start_us":602973,"threads_started":2,"wal-append.queue_time_us":232828}
I20260709 13:05:27.038688 21851 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.329s
I20260709 13:05:27.323683 22046 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.296366 22034 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.629s	user 0.001s	sys 0.000s
I20260709 13:05:27.303565 21981 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:27.304713 22033 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:05:27.038514 21915 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.304219 22029 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:05:27.377202 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:27.333019 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:27.336910 22044 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.337345 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.337479 22019 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.337626 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.526355 15157 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:27.338709 22041 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.412909 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1615) took 2068 ms. Trace:
I20260709 13:05:27.537132 15084 rpcz_store.cc:276] 0709 13:05:25.344109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.206560 (+862451us) service_pool.cc:224] Handling call
0709 13:05:26.207057 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:26.319357 (+112300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.412560 (+1093203us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540011b2d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:27.412813 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256396672}
W20260709 13:05:27.414500 21962 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.114s	user 0.008s	sys 0.002s
W20260709 13:05:27.416636 15088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2489) took 1753 ms (client timeout 2205 ms). Trace:
I20260709 13:05:27.568189 22041 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:27.419663 15231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2685) took 2154 ms (client timeout 2176 ms). Trace:
I20260709 13:05:27.345829 22036 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.423117 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3108) took 1995 ms. Trace:
I20260709 13:05:27.423808 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2672) took 2215 ms. Trace:
I20260709 13:05:27.376716 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:27.027714 21930 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:27.047391 21970 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:27.662931 15228 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 11.
I20260709 13:05:27.466404 15845 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:27.408303 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:27.470003 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.650362 21711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.012s	sys 0.032s
W20260709 13:05:27.489431 15847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.048s	sys 0.121s
W20260709 13:05:27.494300 15087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.055s	sys 0.078s
W20260709 13:05:27.497852 15202 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.052s	sys 0.074s
W20260709 13:05:27.503135 15236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.044s	sys 0.088s
W20260709 13:05:27.668978 15847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.048s	sys 0.121s
W20260709 13:05:27.669114 15087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.055s	sys 0.078s
W20260709 13:05:27.669395 15236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.044s	sys 0.088s
I20260709 13:05:27.670089 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2421) took 1174 ms. Trace:
I20260709 13:05:27.667423 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:27.669183 15202 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.052s	sys 0.074s
I20260709 13:05:27.670502 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:27.671211 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 787627us
I20260709 13:05:27.671370 15847 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:27.506136 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:27.672369 15846 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
W20260709 13:05:27.496536 21895 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.174s	user 0.011s	sys 0.000s
I20260709 13:05:27.562630 15230 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:05:27.661106 15088 rpcz_store.cc:269] 0709 13:05:25.663510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.747900 (+ 84390us) service_pool.cc:224] Handling call
0709 13:05:27.416449 (+1668549us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54001b17d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.416569 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439500544}
W20260709 13:05:27.673434 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.044296 21194 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.304446 21753 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.216s
W20260709 13:05:27.573058 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.506724 21883 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:27.505863 21931 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.365943 22037 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.503252 21889 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
W20260709 13:05:27.617956 22034 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:05:27.506861 21833 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 13:05:27.438012 22047 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.652307 22050 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:27.676888 21869 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.385056 21881 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:27.645356 21543 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.258s	user 0.011s	sys 0.002s
I20260709 13:05:27.362697 21042 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:27.417773 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1587) took 1525 ms. Trace:
I20260709 13:05:27.618770 22052 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
W20260709 13:05:27.660117 15231 rpcz_store.cc:269] 0709 13:05:25.265564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.473360 (+207796us) service_pool.cc:224] Handling call
0709 13:05:27.419488 (+1946128us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.419602 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703457536}
W20260709 13:05:27.420634 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.563524 22019 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.662451 15234 rpcz_store.cc:276] 0709 13:05:25.207956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.232425 (+ 24469us) service_pool.cc:224] Handling call
0709 13:05:25.445177 (+212752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.495542 (+ 50365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.423619 (+1928077us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540005bc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.423751 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488587136}
I20260709 13:05:27.537278 22044 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:27.481853 15840 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.000s
I20260709 13:05:27.662333 15163 rpcz_store.cc:276] 0709 13:05:25.427233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.427719 (+   486us) service_pool.cc:224] Handling call
0709 13:05:26.054795 (+627076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.096144 (+ 41349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.422887 (+1326743us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540033f250. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.423045 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409801088}
I20260709 13:05:27.697427 21883 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.568940 21962 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.679651 15079 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:27.658149 22033 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:27.689392 21931 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.765458 22033 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:27.766100 21915 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:27.766446 22033 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:27.767918 21931 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:27.673156 21711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.012s	sys 0.032s
I20260709 13:05:27.761703 21929 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:05:27.497671 22046 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.723889 22049 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:27.521903 21981 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:27.711684 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 390298us
I20260709 13:05:27.579483 15841 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:27.756273 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.759431 15840 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:27.796741 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1666) took 1219 ms. Trace:
I20260709 13:05:27.770854 21042 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:05:27.797222 15840 rpcz_store.cc:276] 0709 13:05:26.577105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.581870 (+  4765us) service_pool.cc:224] Handling call
0709 13:05:27.796674 (+1214804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:27.798543 15848 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [15.11-16.12]
W20260709 13:05:27.675823 21978 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 2.620s	user 0.002s	sys 0.011s
I20260709 13:05:27.711498 15161 rpcz_store.cc:276] 0709 13:05:25.892174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.283055 (+390881us) service_pool.cc:224] Handling call
0709 13:05:26.478318 (+195263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.488042 (+  9724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.417568 (+929526us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540037e950. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:27.417710 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448231296}
I20260709 13:05:27.767931 22033 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:27.769487 15846 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:27.689011 15847 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:27.760236 21928 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.316s	user 0.013s	sys 0.000s
I20260709 13:05:27.705785 15842 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.804036 15840 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:05:27.771337 21641 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 1.394s	user 0.011s	sys 0.002s
I20260709 13:05:27.799762 15126 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:27.680537 21895 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.802599 15844 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:27.807163 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:27.809338 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:27.708324 22050 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:27.760113 21543 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:27.819120 15154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.079s	sys 0.102s
I20260709 13:05:27.717145 21869 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:27.821547 15154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.079s	sys 0.102s
I20260709 13:05:27.774477 21883 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.826974 22052 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:27.827896 22052 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:27.830601 15843 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:27.804951 22049 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:27.808831 21833 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:27.812960 21641 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
W20260709 13:05:27.833930 21878 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.893s	user 0.008s	sys 0.002s
I20260709 13:05:27.836621 21883 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:27.838340 21981 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:27.842392 21869 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:27.842278 22046 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:27.855691 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 178422us
I20260709 13:05:27.859048 21883 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:27.844302 21833 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:27.853313 22057 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:27.856446 21878 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.884618 21878 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:27.869894 21976 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.965s	user 0.008s	sys 0.003s
I20260709 13:05:27.864688 21194 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:27.872298 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:27.867805 15201 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.072s (SENT)
I20260709 13:05:27.895174 22049 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.897058 21543 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:27.898197 21895 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:27.900014 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:27.858631 21641 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
W20260709 13:05:27.912992 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 225331us
I20260709 13:05:27.890134 22057 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:27.916750 21889 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.963197 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.916671 22059 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:27.919950 21962 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:27.947710 22021 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.380s	user 0.013s	sys 0.000s
I20260709 13:05:27.961798 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:27.964334 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 150537us
I20260709 13:05:27.903553 21976 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.968386 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.970094 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.971004 21994 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:27.971112 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.972793 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.974553 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.018828 15157 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:27.976234 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:27.956534 22064 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:27.958997 22044 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:27.959542 21895 consensus_queue.cc:1059] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:05:27.907846 21833 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:27.987597 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1585) took 1692 ms. Trace:
I20260709 13:05:27.989741 21194 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.999315 21930 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.001682 15234 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:28.002556 15231 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:28.003170 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.035238 15230 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:28.005951 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:28.009882 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:28.010715 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:27.965576 21928 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.000625 22036 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:27.670433 15236 rpcz_store.cc:276] 0709 13:05:26.496001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.533571 (+ 37570us) service_pool.cc:224] Handling call
0709 13:05:26.538456 (+  4885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.538638 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:26.799650 (+261012us) log.cc:844] Serialized 14 byte log entry
0709 13:05:27.669767 (+870117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:27.669816 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:27.669856 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.669921 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:27.670033 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":399042,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":317,"thread_start_us":1103517,"threads_started":2}
I20260709 13:05:28.026700 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:28.030303 15159 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:28.030234 15842 rpcz_store.cc:276] 0709 13:05:26.295567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.295947 (+   380us) service_pool.cc:224] Handling call
0709 13:05:26.310682 (+ 14735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:27.804316 (+1493634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.987308 (+182992us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54007e4354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:27.987525 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3201145600}
I20260709 13:05:28.027771 22064 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.017076 22021 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.040980 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:28.042047 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 369255us
I20260709 13:05:28.045625 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:28.046468 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 178668us
I20260709 13:05:28.031908 22050 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.048858 21895 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:28.038877 21930 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.049364 21895 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.049855 21895 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:28.047079 22064 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.047219 21994 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:27.967178 21889 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:28.050988 21973 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.136s	user 0.006s	sys 0.004s
I20260709 13:05:28.051293 21994 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:28.051508 21973 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.052274 21994 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:28.038563 21833 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:28.053508 15158 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:27.968312 15128 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:28.054255 15162 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:28.057663 21962 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:28.058403 21962 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:28.056715 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103318us
W20260709 13:05:28.071164 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.073944 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 201575us
I20260709 13:05:28.036094 21194 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.063277 21948 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.083772 22049 consensus_queue.cc:1059] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:28.084892 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:28.086302 22023 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.086807 22023 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.087743 22029 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:05:28.088739 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.090770 21931 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:28.091393 21931 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.092298 22041 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.092756 21927 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:28.093416 21927 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.093640 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.093864 21927 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:28.094300 15229 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:28.095263 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.095989 15229 raft_consensus.cc:2410] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:28.097344 15235 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:28.098301 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.098837 15235 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:28.099603 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.100174 21889 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.072000 21869 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.092826 22041 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.101400 22041 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.102061 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.103144 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.103574 22041 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.103796 15233 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:28.105777 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.105841 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:28.106927 21922 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:28.107156 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.107419 21922 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.107883 21922 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:28.108232 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.110554 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.111186 15232 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:28.112227 15232 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:28.067580 22057 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.117331 15157 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:28.047531 21928 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.120152 21928 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.120909 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.121539 15236 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:28.122427 21962 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:28.126748 15161 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:28.127686 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52250.7us
I20260709 13:05:28.128995 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:28.100204 15235 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:28.096716 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.084370 21948 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.104600 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.154487 15233 raft_consensus.cc:2393] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:28.101176 21869 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.139987 21962 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:28.140937 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:28.141570 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.142357 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3151) took 1216 ms. Trace:
I20260709 13:05:28.164592 15087 rpcz_store.cc:276] 0709 13:05:26.926032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.926233 (+   201us) service_pool.cc:224] Handling call
0709 13:05:26.926663 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.926932 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:26.927334 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 13:05:27.939969 (+1012635us) log.cc:844] Serialized 14 byte log entry
0709 13:05:27.953895 (+ 13926us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.141601 (+187706us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b180045f160. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3f59ad70 00007f0d3f58e293 00007f0d3f583cea 00007f0d3f58423a 00007f0d3f9eea2a 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:05:28.141847 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:28.141900 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:28.141947 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.142014 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.142295 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":164516,"prepare.queue_time_us":206,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":23836,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":17517,"spinlock_wait_cycles":7895808,"thread_start_us":1011936,"threads_started":2,"wal-append.queue_time_us":927722}
I20260709 13:05:28.131886 21711 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.143517 15236 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:28.143959 15229 raft_consensus.cc:2376] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17. Re-sending same reply.
I20260709 13:05:28.148352 22019 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.150509 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.152364 21930 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.153220 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.155304 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.145464 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:28.124796 21931 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.169960 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.126423 21731 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.174909 15160 raft_consensus.cc:2368] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:28.176812 15848 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:28.163679 22055 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:05:28.191042 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.193526 15844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:28.204578 15844 raft_consensus.cc:606] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:28.204968 15844 raft_consensus.cc:614] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:28.196470 15232 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:28.205965 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:28.201967 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.211526 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1587) took 1907 ms. Trace:
I20260709 13:05:28.212158 15843 rpcz_store.cc:276] 0709 13:05:26.304104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.304933 (+   829us) service_pool.cc:224] Handling call
0709 13:05:26.322863 (+ 17930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:27.831320 (+1508457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.200118 (+368798us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400852ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:28.211454 (+ 11336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3466802944}
I20260709 13:05:28.163223 21929 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:28.222398 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.224167 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:28.227097 15843 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:28.227944 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90967.8us
W20260709 13:05:28.221547 15051 connection.cc:256] RPC call timeout handler was delayed by -1.4209776760000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 13:05:28.230291 15051 connection.cc:256] RPC call timeout handler was delayed by -1.4130596779999678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 13:05:28.234093 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1597) took 1157 ms. Trace:
I20260709 13:05:28.234558 15843 rpcz_store.cc:276] 0709 13:05:27.070185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.226545 (+1156360us) service_pool.cc:224] Handling call
0709 13:05:28.226908 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.227494 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.227695 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:28.239059 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.239753 15084 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:28.240892 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.241123 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.241484 15083 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:28.241992 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.242116 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.242461 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.242643 15082 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:28.243162 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.243360 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:28.243522 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.244203 15085 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:28.244221 22041 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:28.246373 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.248234 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.239862 15086 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:05:28.249171 15086 pending_rounds.cc:85] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 12
I20260709 13:05:28.249459 15086 pending_rounds.cc:107] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:28.249972 15086 raft_consensus.cc:2889] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.246343 22041 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:05:28.252606 22041 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:28.249373 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.255591 15085 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:28.256407 22041 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:28.258260 22041 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:28.259172 22041 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:28.259552 15051 leader_election.cc:400] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:28.259755 22041 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:28.260540 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:28.261004 22041 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.262452 15051 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:28.262634 22041 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:28.263258 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:28.267666 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:28.259143 15082 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:28.268903 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 123473us
I20260709 13:05:28.205258 22061 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.301234 15080 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:28.304227 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:28.305662 15841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1616) took 1778 ms (client timeout 2022 ms). Trace:
W20260709 13:05:28.306213 15841 rpcz_store.cc:269] 0709 13:05:26.526993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.527370 (+   377us) service_pool.cc:224] Handling call
0709 13:05:28.305436 (+1778066us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b5400822154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:28.305592 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213213952}
I20260709 13:05:28.221952 21928 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.226944 21731 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.226137 21929 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.315836 15163 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:28.325779 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.317682 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.326599 15087 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:28.182896 21711 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.341686 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3112) took 2911 ms. Trace:
I20260709 13:05:28.342136 21948 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.342937 22085 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:28.343807 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.294963 22019 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.345641 22047 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.295414 22059 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:28.346992 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:28.347937 21939 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.909s	user 0.009s	sys 0.003s
I20260709 13:05:28.232774 21930 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.202666 21895 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.325577 21929 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:28.355783 22071 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:28.356992 22037 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.357161 21869 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.318290 22061 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.358361 22089 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:28.360900 22021 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.359098 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.359164 22084 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:28.360862 22064 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.325162 21928 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.362370 15155 raft_consensus.cc:3055] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:28.377458 21948 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:28.364408 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:28.375944 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 112770us
I20260709 13:05:28.374621 15160 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
W20260709 13:05:28.422737 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58408.5us
I20260709 13:05:28.421715 15155 raft_consensus.cc:740] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:28.392463 22047 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:28.396389 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.398306 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.396203 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.429277 15083 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:28.429608 15087 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:28.401808 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:28.406454 15162 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:28.411967 22071 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.371860 22085 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:28.366863 15154 rpcz_store.cc:276] 0709 13:05:25.430723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.182961 (+752238us) service_pool.cc:224] Handling call
0709 13:05:26.197743 (+ 14782us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:26.235548 (+ 37805us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:26.269688 (+ 34140us) log.cc:844] Serialized 14 byte log entry
0709 13:05:26.951047 (+681359us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b18003fb800. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a 00007f0d3f9eea2a
0709 13:05:27.900784 (+949737us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b18003fb800. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f73f3 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:28.038272 (+137488us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:28.041506 (+  3234us) log.cc:844] Serialized 52 byte log entry
0709 13:05:28.041802 (+   296us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:28.041989 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.042757 (+   768us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.341485 (+298728us) raft_consensus.cc:1685] finished
0709 13:05:28.341507 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.341628 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34087,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":40487,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":32477,"spinlock_wait_cycles":15110912,"thread_start_us":1083137,"threads_started":2,"wal-append.queue_time_us":1648656}
W20260709 13:05:28.390475 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 123637us
W20260709 13:05:28.426712 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.366899 21731 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:28.398419 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.429483 15080 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:28.427779 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:28.438855 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:28.371593 21962 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:05:28.406507 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 138799us
I20260709 13:05:28.449674 15082 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:28.376931 22019 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.444698 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:28.436687 15228 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:28.425457 21931 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:28.427595 22041 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:28.457710 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.392589 21939 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.450417 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:28.442899 15155 consensus_queue.cc:260] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:28.454762 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.456698 15229 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:05:28.467967 15229 pending_rounds.cc:85] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 10
I20260709 13:05:28.468169 15230 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:28.468255 15229 pending_rounds.cc:107] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:05:28.458791 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.469329 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.468647 15229 raft_consensus.cc:2889] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.469987 15231 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.469831 15235 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:28.464915 15233 raft_consensus.cc:2376] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16. Re-sending same reply.
I20260709 13:05:28.408397 21930 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.469281 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.474361 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.475148 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.475705 15230 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:28.431402 22047 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.392266 22059 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.488488 15231 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:28.469246 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.469892 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.479375 15083 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:28.481873 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.490322 15233 raft_consensus.cc:2376] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18. Re-sending same reply.
I20260709 13:05:28.490712 15082 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:28.490715 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.491247 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:28.490103 15234 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:28.441665 22071 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:28.482784 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.431563 22036 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:05:28.450160 22098 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.547431 22098 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.550213 22098 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.487911 15230 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:28.480661 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:28.472318 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48428.3us
I20260709 13:05:28.509563 15088 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:28.418973 22037 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.510609 15083 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:28.554418 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.555472 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.556129 15083 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:28.513442 15845 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1625) took 1473 ms (client timeout 1602 ms). Trace:
W20260709 13:05:28.556993 15845 rpcz_store.cc:269] 0709 13:05:27.039977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.056004 (+ 16027us) service_pool.cc:224] Handling call
0709 13:05:28.513174 (+1457170us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400580054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:28.513367 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368448768}
I20260709 13:05:28.558151 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:28.558679 15845 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.437146 22027 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:05:28.491964 15233 raft_consensus.cc:2368] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:28.526214 15084 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:28.572402 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:28.556798 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:28.545814 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101178us
I20260709 13:05:28.542091 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:28.573774 15083 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:28.583472 15234 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 12.
I20260709 13:05:28.577688 22037 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.573122 15154 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:28.587173 15845 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
W20260709 13:05:28.572785 22033 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.766s	user 0.005s	sys 0.004s
I20260709 13:05:28.591187 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.593792 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.594936 15083 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:28.597163 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:28.599923 15083 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:28.600891 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:28.598862 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.609010 15845 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:28.611676 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.616892 21641 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.617658 21641 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.621006 22029 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.621538 22029 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:28.613776 15845 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:28.628747 21641 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.637789 22029 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.639988 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.658965 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.664678 22092 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:28.665917 22086 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
I20260709 13:05:28.701805 15845 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:28.487394 22046 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:28.486678 22047 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:28.686158 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 205567us
I20260709 13:05:28.721531 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:28.488451 21915 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.199s
I20260709 13:05:28.701059 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.481379 22096 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.212s
I20260709 13:05:28.695912 22098 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:28.686756 21948 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.729512 15079 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:28.718747 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.710278 15083 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:05:28.716708 15156 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:28.667305 21922 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.758855 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.759948 15083 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:28.761214 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.711633 22046 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.762127 15845 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:28.765771 22087 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:28.766676 15086 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:28.767518 22064 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.728803 22082 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:28.764425 22081 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:05:28.770951 15843 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:28.772769 21928 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:28.773593 22097 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:28.776940 22046 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:28.782922 22058 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.813s	user 0.004s	sys 0.000s
I20260709 13:05:28.784427 22044 raft_consensus.cc:3055] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:28.794894 15235 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:28.798616 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:28.801591 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:28.849952 15159 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:28.764562 22095 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:28.756897 22098 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.799435 22087 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.797756 22082 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.797786 22027 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:28.886207 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:28.901947 15840 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:28.761078 21948 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:28.810401 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:28.886499 21927 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.003s
I20260709 13:05:28.804556 21895 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:28.929837 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:28.931588 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 380034us
I20260709 13:05:28.931842 15844 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:28.932480 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:28.937697 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 487325us
I20260709 13:05:28.939556 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:28.847007 22041 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:28.942461 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 132286us
I20260709 13:05:28.865236 22100 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.865315 22098 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:28.962102 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:28.964427 15085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.018s	sys 0.039s
I20260709 13:05:28.856118 22044 raft_consensus.cc:740] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:28.897472 22087 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:28.808936 21711 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.890369 22082 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.991341 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:28.991986 15840 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:28.971179 15051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.024s	sys 0.064s
I20260709 13:05:28.966264 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1594) took 2063 ms. Trace:
W20260709 13:05:28.978029 15085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.022s	sys 0.039s
I20260709 13:05:28.847285 22029 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:05:28.809897 22068 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 13:05:28.900914 21731 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:28.956276 22088 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.983983 21922 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.993110 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1598) took 1488 ms. Trace:
I20260709 13:05:28.976107 21931 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:28.961158 21994 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:29.003234 15848 rpcz_store.cc:276] 0709 13:05:27.504870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.229673 (+724803us) service_pool.cc:224] Handling call
0709 13:05:28.312409 (+ 82736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.315224 (+  2815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.315265 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.327560 (+ 12295us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.403695 (+ 76135us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:28.403735 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.403820 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.992871 (+589051us) raft_consensus.cc:1685] finished
0709 13:05:28.992894 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.993054 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2353,"raft.queue_time_us":2961,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":32645,"thread_start_us":39931,"threads_started":1,"wal-append.queue_time_us":95844}
I20260709 13:05:28.994294 15839 rpcz_store.cc:276] 0709 13:05:26.902586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.933825 (+ 31239us) service_pool.cc:224] Handling call
0709 13:05:26.998096 (+ 64271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.059745 (+ 61649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.059789 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.067001 (+  7212us) log.cc:844] Serialized 26 byte log entry
0709 13:05:27.806985 (+739984us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:27.853544 (+ 46559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.853679 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.782191 (+928512us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:28.957544 (+175353us) raft_consensus.cc:1685] finished
0709 13:05:28.957572 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.966172 (+  8600us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61113,"raft.queue_time_us":69436,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9912,"spinlock_wait_cycles":77734656,"thread_start_us":438318,"threads_started":1,"wal-append.queue_time_us":596062}
W20260709 13:05:29.003309 15080 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.021s	sys 0.040s
W20260709 13:05:29.012969 15080 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.021s	sys 0.040s
I20260709 13:05:28.961194 22061 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:28.964906 22064 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:28.994275 15051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.025s	sys 0.064s
W20260709 13:05:29.015007 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 587261us
I20260709 13:05:28.936643 22027 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.964262 21939 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.983392 22033 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:29.002552 22103 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:28.974869 21942 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 13:05:29.016502 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 286117us
W20260709 13:05:29.025367 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 425049us
I20260709 13:05:28.984529 21930 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:28.857327 21928 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:29.015734 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:29.015640 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 76151.2us
I20260709 13:05:28.944907 21927 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.996428 22021 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:29.029484 22103 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.008921 22088 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:28.963707 21881 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:05:28.975035 21641 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:05:28.954473 15844 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:29.038558 15840 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:29.043376 21930 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:29.043830 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:29.042304 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:29.076920 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.078511 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:28.990907 21895 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:05:29.011507 21931 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:29.016872 22101 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:29.048393 15054 leader_election.cc:400] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:29.087160 15054 leader_election.cc:403] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.044045 22096 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:28.976680 22100 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.053725 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.110954 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.019800 21994 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:29.033000 22033 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.042047 21939 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:29.059908 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.004s	sys 0.019s
I20260709 13:05:29.039116 22107 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:29.080829 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:29.104936 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 45174.3us
I20260709 13:05:29.064463 22021 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:29.038067 22044 consensus_queue.cc:260] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.047048 21928 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.089028 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.088310 21931 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.049292 22041 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:29.065866 22103 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:29.126094 15848 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:29.112722 15159 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:29.120267 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 3122) took 1079 ms. Trace:
I20260709 13:05:29.126660 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.122435 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1494) took 1049 ms. Trace:
W20260709 13:05:29.120419 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.005s	sys 0.019s
I20260709 13:05:29.121263 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1629) took 1156 ms. Trace:
W20260709 13:05:29.141989 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 653515us
I20260709 13:05:29.142025 15841 rpcz_store.cc:276] 0709 13:05:27.964787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.318574 (+353787us) service_pool.cc:224] Handling call
0709 13:05:28.318939 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.319318 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.319357 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.363999 (+ 44642us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.859368 (+495369us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:28.859430 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.859545 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.120722 (+261177us) raft_consensus.cc:1685] finished
0709 13:05:29.120750 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.121195 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13306,"raft.queue_time_us":494,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":26371,"thread_start_us":443106,"threads_started":1,"wal-append.queue_time_us":682681}
I20260709 13:05:29.128427 15201 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.145714 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.146320 15081 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:29.146659 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:29.147189 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:29.147847 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.148401 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.118695 22113 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 13:05:29.148825 22111 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:29.149359 22064 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:29.149652 22111 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:29.150082 22108 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.150847 22064 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:29.151393 22064 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:29.130853 22104 log.cc:927] Time spent T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:29.153013 15201 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.130234 15840 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:29.113263 22102 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:05:29.117944 22027 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.117460 22068 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:29.140120 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1603) took 1162 ms. Trace:
I20260709 13:05:29.154901 15847 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:29.154764 15161 rpcz_store.cc:276] 0709 13:05:27.977201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.010481 (+ 33280us) service_pool.cc:224] Handling call
0709 13:05:29.139938 (+1129457us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b540033fc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:29.140062 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204978048}
I20260709 13:05:29.114255 22033 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.141103 15156 rpcz_store.cc:276] 0709 13:05:28.040474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.040628 (+   154us) service_pool.cc:224] Handling call
0709 13:05:29.120055 (+1079427us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b540037ee54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:29.120186 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252645248}
I20260709 13:05:29.141433 15844 raft_consensus.cc:2393] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:29.151218 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.114970 22088 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.112979 22096 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:29.141419 15237 rpcz_store.cc:276] 0709 13:05:28.073158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.107987 (+ 34829us) service_pool.cc:224] Handling call
0709 13:05:29.122253 (+1014266us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b540005c654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:29.122371 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223315200}
I20260709 13:05:29.154225 22068 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:29.136129 22044 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:29.140568 22075 raft_consensus.cc:2764] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:05:29.156193 21948 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:29.136080 22115 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.155256 21939 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.205775 15201 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:29.207330 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.207871 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:29.204313 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 125909us
I20260709 13:05:29.152424 22064 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.191537 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:29.156358 21930 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:29.208933 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.166059 22106 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:29.223255 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.168339 22107 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:29.187834 22118 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:29.113991 21994 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.203467 22113 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:29.203907 22114 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:29.237540 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:29.234818 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 185429us
I20260709 13:05:29.223248 21711 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:29.223137 21930 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.244769 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 111977us
I20260709 13:05:29.231489 21948 raft_consensus.cc:2764] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:05:29.247828 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 202037us
I20260709 13:05:29.223457 22106 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:29.249838 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:29.225983 21973 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.236922 21939 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:29.230723 22118 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:29.295938 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:29.248167 21930 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:29.249925 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:29.279174 15230 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.012s	sys 0.003s
I20260709 13:05:29.292744 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:29.260201 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 326538us
W20260709 13:05:29.297358 15230 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.013s	sys 0.003s
I20260709 13:05:29.297595 15086 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:29.234493 22101 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:29.298046 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:29.247984 22102 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:29.151129 22111 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.299322 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.253064 22113 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:29.242511 22080 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:29.299321 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:29.297361 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47540.2us
I20260709 13:05:29.302743 22102 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.303128 15085 raft_consensus.cc:1240] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: [13.12-15.13]
I20260709 13:05:29.257476 22108 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:29.306048 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:29.296001 21929 consensus_meta.cc:287] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.878s	user 0.012s	sys 0.000s
I20260709 13:05:29.224951 22109 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:05:29.307466 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.310299 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.314474 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:29.314659 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:29.315418 15233 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:29.258215 21711 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:29.334095 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:29.334906 15233 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:29.307461 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:29.317607 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 148837us
I20260709 13:05:29.328805 15085 raft_consensus.cc:2410] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:29.323308 15202 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:29.284132 22106 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.337251 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.340010 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:29.341918 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:29.342852 15235 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:29.308480 22113 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.344280 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:29.350620 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 159181us
I20260709 13:05:29.307201 22100 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:29.350760 15085 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:29.316257 22117 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:29.311844 22105 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:29.347747 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 124640us
I20260709 13:05:29.325512 22112 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.339344 21948 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:29.298944 22101 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.301196 21939 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:29.351637 22071 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.003s
W20260709 13:05:29.380055 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 53734.1us
I20260709 13:05:29.374223 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.344990 22122 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.344635 22061 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.390089 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:29.403461 15840 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:29.353745 22113 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:29.251562 22068 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:29.400988 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 151247us
I20260709 13:05:29.402204 15051 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:29.362110 21928 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.354141 22126 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.364215 22129 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.366986 22117 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.350112 21922 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:05:29.364604 22100 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.332207 21929 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:29.417510 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:29.420413 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2689) took 1287 ms. Trace:
I20260709 13:05:29.372252 22118 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:29.320449 22107 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.413249 22122 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.429041 15233 raft_consensus.cc:2468] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:29.382164 22105 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:29.426685 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:29.396858 22108 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:29.447871 15162 rpcz_store.cc:276] 0709 13:05:28.132692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.137478 (+  4786us) service_pool.cc:224] Handling call
0709 13:05:28.137816 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.430455 (+292639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.420160 (+989705us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001d8654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:29.420352 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617350912}
I20260709 13:05:29.447120 21928 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:29.422473 22120 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 13:05:29.452850 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 129663us
I20260709 13:05:29.407666 21948 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:29.325891 22102 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.446219 22117 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:29.455617 21194 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.460637 22105 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.414906 22061 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.407135 22071 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.455978 22144 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:29.460453 22108 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.460618 21928 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:29.390089 22112 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:29.486711 22119 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
I20260709 13:05:29.468817 22124 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.490164 22027 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:29.511052 22121 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:05:29.453413 22100 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.529036 15237 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:29.517035 21928 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:29.511106 22108 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:29.510605 22138 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:29.458024 21711 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.521577 21976 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
W20260709 13:05:29.569409 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 142822us
I20260709 13:05:29.523394 21881 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:29.574819 21881 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:29.524659 22144 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:29.585713 21939 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.590955 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2440) took 1131 ms. Trace:
I20260709 13:05:29.595801 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2441) took 1136 ms. Trace:
I20260709 13:05:29.596228 15236 rpcz_store.cc:276] 0709 13:05:28.459780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.542870 (+ 83090us) service_pool.cc:224] Handling call
0709 13:05:28.694740 (+151870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.703376 (+  8636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.703427 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.740175 (+ 36748us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.045644 (+305469us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:29.045684 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.045762 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.595568 (+549806us) raft_consensus.cc:1685] finished
0709 13:05:29.595589 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.595739 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8217,"raft.queue_time_us":169105,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":22885,"thread_start_us":260340,"threads_started":1,"wal-append.queue_time_us":425597}
I20260709 13:05:29.564196 22138 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.591385 15232 rpcz_store.cc:276] 0709 13:05:28.459505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.542729 (+ 83224us) service_pool.cc:224] Handling call
0709 13:05:28.691146 (+148417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:28.730229 (+ 39083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.730284 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:28.769603 (+ 39319us) log.cc:844] Serialized 52 byte log entry
0709 13:05:29.073777 (+304174us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:29.228595 (+154818us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.229397 (+   802us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.354490 (+125093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.354718 (+   228us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.590701 (+235983us) raft_consensus.cc:1685] finished
0709 13:05:29.590720 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.590893 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35998,"prepare.queue_time_us":506673,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":215293,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":30440,"thread_start_us":447817,"threads_started":3,"wal-append.queue_time_us":412412}
I20260709 13:05:29.553535 21948 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:29.590593 22047 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.000s
I20260709 13:05:29.616276 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:29.523746 22124 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.595204 22125 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.636881 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3170) took 1239 ms. Trace:
I20260709 13:05:29.637389 15087 rpcz_store.cc:276] 0709 13:05:28.397265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.431244 (+ 33979us) service_pool.cc:224] Handling call
0709 13:05:28.451761 (+ 20517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:28.712394 (+260633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.712446 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:28.713162 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 13:05:29.177941 (+464779us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:29.334799 (+156858us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.338694 (+  3895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.338828 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.636621 (+297793us) raft_consensus.cc:1685] finished
0709 13:05:29.636644 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.636798 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7209,"prepare.queue_time_us":235150,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":596627,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7262,"spinlock_wait_cycles":15925504,"thread_start_us":626335,"threads_started":3,"wal-append.queue_time_us":396472}
W20260709 13:05:29.647336 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.618716 21753 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.649288 21753 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.649863 22144 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.651911 21753 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:29.655753 22082 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.000s
I20260709 13:05:29.656194 22082 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.593961 22140 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
W20260709 13:05:29.628191 22127 log.cc:927] Time spent T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:05:29.662021 22100 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:29.663267 22071 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.663506 22112 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.612573 22138 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:29.677942 22125 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:29.681084 22120 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:29.681449 21929 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:29.681309 22124 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.577519 21881 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.688869 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:29.718803 22098 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.719s	user 0.013s	sys 0.000s
W20260709 13:05:29.694072 22046 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.003s
W20260709 13:05:29.708140 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 370935us
I20260709 13:05:29.689600 22061 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.618693 22047 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:29.713083 22103 consensus_meta.cc:287] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.005s
W20260709 13:05:29.693710 22087 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.698s	user 0.009s	sys 0.002s
I20260709 13:05:29.660617 22106 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.697073 22122 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:29.735610 15087 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:29.662649 22033 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.717448 21994 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:05:29.725728 21931 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.002s
I20260709 13:05:29.765941 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:29.747700 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.750820 22139 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:29.755426 22131 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.761603 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.764914 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:29.770375 15085 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:29.767117 15083 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:29.769665 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1622) took 1321 ms. Trace:
I20260709 13:05:29.759400 22023 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.751780 22046 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.771507 21931 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.772960 22082 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.774142 15841 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:29.770970 15080 rpcz_store.cc:276] 0709 13:05:28.449616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.450906 (+  1290us) service_pool.cc:224] Handling call
0709 13:05:28.451311 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.013470 (+562159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.013512 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.042944 (+ 29432us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.223411 (+180467us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:29.223454 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.223532 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.769446 (+545914us) raft_consensus.cc:1685] finished
0709 13:05:29.769466 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.769608 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":444619,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":44137,"thread_start_us":708713,"threads_started":2,"wal-append.queue_time_us":285219}
I20260709 13:05:29.783818 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:29.792444 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.747947 22098 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:29.797914 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50300.3us
I20260709 13:05:29.817394 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.818462 22098 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.818648 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.820397 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.768981 22139 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:29.784410 22114 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:05:29.798982 15810 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:29.821529 22114 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:29.821745 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.822495 22161 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:05:29.822877 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:29.823562 22161 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:29.823892 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:29.824177 22100 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:29.805344 22122 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:29.806134 22125 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:29.824810 21930 consensus_meta.cc:287] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.003s
I20260709 13:05:29.825168 22122 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:29.804580 15087 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:29.808079 22106 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:29.809461 22044 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.003s
I20260709 13:05:29.762060 22087 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:29.837493 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 248251us
I20260709 13:05:29.811028 22033 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.812538 22105 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.797593 22061 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:29.814340 22047 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.829901 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.830847 15201 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:29.762389 22103 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.827603 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:29.825580 22113 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.840096 22102 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.840631 22037 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:29.842372 22087 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.853802 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:29.834802 22122 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:29.842842 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.826952 22112 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:29.829691 21976 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.843593 22021 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:29.823931 22161 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:29.918895 22044 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.919579 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1490) took 1922 ms. Trace:
W20260709 13:05:29.919639 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 92119.4us
I20260709 13:05:29.921314 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:29.945078 15154 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:29.921875 22085 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.926299 22090 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:29.928462 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1607) took 1079 ms. Trace:
I20260709 13:05:29.949194 15848 rpcz_store.cc:276] 0709 13:05:28.848956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.110646 (+261690us) service_pool.cc:224] Handling call
0709 13:05:29.928258 (+817612us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54006cdc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:29.928396 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33129728}
I20260709 13:05:29.949544 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.929944 22037 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:29.956787 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:29.938486 22057 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:29.825001 22100 raft_consensus.cc:2804] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:29.854756 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.934401 15162 raft_consensus.cc:2368] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:29.934722 15844 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:29.944706 22161 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:29.944617 15228 rpcz_store.cc:276] 0709 13:05:27.997047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.003636 (+  6589us) service_pool.cc:224] Handling call
0709 13:05:28.003953 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.453684 (+449731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.919299 (+1465615us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540005c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:29.919510 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994570496}
W20260709 13:05:29.925607 22123 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.559s	user 0.000s	sys 0.002s
I20260709 13:05:29.946297 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.784158 22131 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.947686 15840 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:29.947855 15081 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:29.954423 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.972580 15160 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:29.972992 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.865664 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:29.955231 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.951347 22090 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:29.960876 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.974910 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.961721 22159 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:29.955400 21976 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:05:29.939040 22089 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:29.936511 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105781us
I20260709 13:05:29.999676 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:29.993165 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:29.836232 21930 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:29.938725 22061 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:29.976050 15156 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:29.973803 15848 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:30.008468 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.951241 22036 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:05:29.977437 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:29.978041 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2445) took 1517 ms. Trace:
W20260709 13:05:29.980336 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.982919 21895 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:30.006624 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:30.052348 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.049178 15229 rpcz_store.cc:276] 0709 13:05:28.460771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.547220 (+ 86449us) service_pool.cc:224] Handling call
0709 13:05:28.547620 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.976228 (+428608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.976276 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.987032 (+ 10756us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.192563 (+205531us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:29.192597 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.192663 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.924924 (+732261us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:29.977724 (+ 52800us) raft_consensus.cc:1685] finished
0709 13:05:29.977746 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.977983 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118208,"raft.queue_time_us":258735,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":37040,"spinlock_wait_cycles":25544960,"thread_start_us":457640,"threads_started":2,"wal-append.queue_time_us":301179}
I20260709 13:05:29.984063 15159 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:05:29.986047 22090 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:29.986781 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:29.989190 22103 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.991451 22131 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:29.996676 15154 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:29.964905 22085 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:29.993381 22159 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.965494 22057 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:29.974429 22100 raft_consensus.cc:697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:30.045843 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:29.915043 22139 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:30.047495 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.067526 15162 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.048785 15844 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:30.062773 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.070880 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.063575 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.064065 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.002903 22112 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:30.066356 15157 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:30.069552 15848 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:30.072721 15229 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:30.073419 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.073570 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.071625 15232 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:30.075385 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.075594 15228 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:30.076004 15229 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.046936 22044 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:29.972137 21928 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.077925 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.072693 15154 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:30.083168 15844 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:05:30.083310 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 76786.6us
I20260709 13:05:30.083775 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.085405 15810 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.085613 15160 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:30.087890 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:30.090350 22117 consensus_meta.cc:287] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:05:30.090843 22117 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.092092 15233 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:30.076754 22023 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:30.093199 22117 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.064759 22159 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.093585 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.093266 22023 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.095506 22023 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.095682 22061 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.098994 22019 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.099406 22112 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:30.099797 22112 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:30.100131 22112 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.106894 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2507) took 1866 ms. Trace:
I20260709 13:05:30.107326 15084 rpcz_store.cc:276] 0709 13:05:28.240276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.240465 (+   189us) service_pool.cc:224] Handling call
0709 13:05:28.240702 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.570518 (+329816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.106645 (+1536127us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54000db6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.106829 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655630720}
I20260709 13:05:30.109233 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3159) took 2434 ms. Trace:
I20260709 13:05:30.109617 15088 rpcz_store.cc:276] 0709 13:05:27.674403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.674526 (+   123us) service_pool.cc:224] Handling call
0709 13:05:27.690988 (+ 16462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.553418 (+862430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.109008 (+1555590us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540034ab54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.109176 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882113920}
I20260709 13:05:30.111295 22068 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.113778 22113 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.115314 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.118042 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1606) took 1269 ms. Trace:
I20260709 13:05:30.118567 15841 rpcz_store.cc:276] 0709 13:05:28.848679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.144377 (+295698us) service_pool.cc:224] Handling call
0709 13:05:29.144765 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.774564 (+629799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.117748 (+343184us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54003a7d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.117971 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446932352}
I20260709 13:05:30.116580 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.122505 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2439) took 1664 ms. Trace:
I20260709 13:05:30.126782 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1601) took 1280 ms. Trace:
I20260709 13:05:30.127357 15842 rpcz_store.cc:276] 0709 13:05:28.846228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.847111 (+   883us) service_pool.cc:224] Handling call
0709 13:05:28.885725 (+ 38614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.140201 (+254476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.140230 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.140273 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:29.140310 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.126509 (+986199us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.126575 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.126715 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584726912}
I20260709 13:05:30.130548 22059 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.132246 22046 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.142725 21931 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.144915 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1628) took 2180 ms. Trace:
I20260709 13:05:30.145826 22159 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.149199 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.150342 15844 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:30.154608 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1625) took 1688 ms. Trace:
I20260709 13:05:30.155195 15083 rpcz_store.cc:276] 0709 13:05:28.466365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.760633 (+294268us) service_pool.cc:224] Handling call
0709 13:05:28.764412 (+  3779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.770947 (+1006535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.154257 (+383310us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400306e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.154525 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305444736}
I20260709 13:05:30.160035 22100 consensus_queue.cc:237] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.162900 15155 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.163427 21948 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:30.164428 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80863.1us
I20260709 13:05:30.151958 15844 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:05:30.145390 15843 rpcz_store.cc:276] 0709 13:05:27.964501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.294655 (+330154us) service_pool.cc:224] Handling call
0709 13:05:28.325264 (+ 30609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.804302 (+479038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.144587 (+1340285us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400771d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:30.144830 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024200576}
I20260709 13:05:30.158731 15162 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:30.169628 15842 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:30.173545 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:30.177774 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:30.178982 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.179284 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:30.179836 15082 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.180493 15083 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:30.181581 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.185333 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.183389 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:30.185930 15087 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.188591 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.189442 15162 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:30.183386 15229 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.123072 15231 rpcz_store.cc:276] 0709 13:05:28.458066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.492990 (+ 34924us) service_pool.cc:224] Handling call
0709 13:05:29.122737 (+629747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.123208 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:29.300566 (+177358us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.406953 (+106387us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.413286 (+  6333us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.413622 (+   336us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:29.413744 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.413889 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.789161 (+375272us) raft_consensus.cc:1685] finished
0709 13:05:29.789197 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.122343 (+333146us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b540005c650. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:30.122450 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":298447,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":14448,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":186,"spinlock_wait_cycles":1446407936,"thread_start_us":215145,"threads_started":2,"wal-append.queue_time_us":346480}
I20260709 13:05:30.195044 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:30.201629 15083 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:30.080197 22090 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:29.997269 22138 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.209174 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:30.006181 22108 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.212185 15201 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:30.212710 15843 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:30.214262 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.214198 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.227617 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.227913 15086 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.228287 15839 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:30.215631 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:30.210515 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:30.216128 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.219662 15844 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:30.229344 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.229887 15839 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.070439 22098 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
W20260709 13:05:30.225306 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.223709 15087 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:30.214052 15202 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:30.203994 22164 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:05:30.209944 22033 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:30.269192 22033 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.269598 22033 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.270176 15054 leader_election.cc:400] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:30.271258 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.273070 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.274163 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:30.239993 15082 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.253320 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.274822 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:30.253962 22089 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:30.276217 15086 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.254292 22108 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.254477 22023 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:30.255179 22170 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:05:30.255360 22131 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:30.257217 15234 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:30.257337 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:30.258180 22044 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:30.116744 22161 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.328455 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.253391 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.329282 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.260443 21976 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:30.210963 21931 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:05:30.208153 22107 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.210619 22057 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.261570 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2448) took 1800 ms. Trace:
I20260709 13:05:30.211169 22126 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:30.263134 15161 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:30.263314 21927 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.207605 21895 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:30.265743 15085 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:30.267037 22138 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.221168 22091 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.277259 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:30.287335 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73377.1us
I20260709 13:05:30.316469 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.318231 15839 raft_consensus.cc:2410] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:30.353387 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.354396 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.355809 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.281642 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:30.321025 15088 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:30.322614 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:30.356827 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 75272.7us
I20260709 13:05:30.357501 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.249156 21881 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:30.330075 15844 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:30.277738 22098 consensus_queue.cc:1243] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:30.364494 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.365387 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.333088 15154 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:30.309031 21942 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.308368 22101 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.328271 22105 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.318351 22085 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:30.349869 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 75162.5us
I20260709 13:05:30.357803 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:30.357511 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.358263 15156 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.385144 15162 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:30.370749 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.374262 15839 raft_consensus.cc:2393] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:30.387969 15839 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:30.335770 15230 rpcz_store.cc:276] 0709 13:05:28.461600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.552497 (+ 90897us) service_pool.cc:224] Handling call
0709 13:05:28.552896 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.298261 (+745365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.298293 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.930184 (+631891us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b2c00032e28. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3e2dda11 00007f0d3e2ed372 00007f0d3fa1fa01 00007f0d3f9e443a 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2
0709 13:05:29.930658 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.046043 (+115385us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:30.046086 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.046159 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.261263 (+215104us) raft_consensus.cc:1685] finished
0709 13:05:30.261293 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.261506 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117655,"raft.queue_time_us":257747,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":18548,"spinlock_wait_cycles":10917248,"thread_start_us":741758,"threads_started":2,"wal-append.queue_time_us":233846}
I20260709 13:05:30.378912 15202 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.354508 15158 raft_consensus.cc:2410] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:30.390205 15839 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:30.392128 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.364641 22021 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:30.396238 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.397017 15085 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:30.343762 21895 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:30.366420 22113 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:30.402956 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.404580 15839 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:30.362347 22057 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:30.419227 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 215031us
I20260709 13:05:30.277094 22122 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.417968 15844 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:30.425745 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:30.348336 21922 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:30.348946 22159 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.416502 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.422349 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.429947 15160 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.407310 22117 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:30.394367 22021 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.340106 22107 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.355476 22126 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:30.430161 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 215969us
I20260709 13:05:30.431602 15162 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.414176 21895 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.409413 22019 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:30.433620 15844 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:30.407121 22085 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:30.383515 21942 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:30.436545 22116 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.427755 21922 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:30.437507 22142 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:05:30.435134 22021 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:30.423243 15810 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.402882 22102 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:05:30.402534 22138 consensus_queue.cc:1059] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:30.421217 22105 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:05:30.403153 22087 consensus_queue.cc:1059] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:30.460028 15844 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:30.436976 22149 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.429441 21973 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:30.461691 15839 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:30.364213 22091 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:30.493031 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 69844.6us
I20260709 13:05:30.494148 22023 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:30.478847 21928 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.505398 22126 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.505854 21928 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:30.506242 22023 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:30.506366 21928 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.506879 22023 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:30.507349 22122 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.507822 22023 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.508050 22149 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.481455 22116 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.510186 21942 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.509972 21922 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.510844 22116 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.438862 22117 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:30.511188 21922 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:30.512032 22117 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.459296 22061 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.513108 22019 raft_consensus.cc:3055] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:30.513418 22176 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:05:30.513264 22061 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.508626 21928 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.515694 22061 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:30.517586 15230 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:30.505411 15201 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:30.518908 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.513628 22019 raft_consensus.cc:740] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:30.521462 15844 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:30.523741 15229 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:30.524880 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.525878 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3128) took 2371 ms. Trace:
I20260709 13:05:30.526054 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:30.526825 15844 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:30.526410 15155 rpcz_store.cc:276] 0709 13:05:28.154687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.361900 (+207213us) service_pool.cc:224] Handling call
0709 13:05:28.362214 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.162314 (+1800100us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b580082ebd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9400aa 00007f0d3f9c2447 00007f0d3f9dafe7 00007f0d3f9dc7de 00007f0d3f9e6b49 00007f0d3f9e7416 00007f0d3f9e3d8f 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4
0709 13:05:30.498434 (+336120us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:30.498483 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.498540 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:30.525535 (+ 26995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.525647 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.525805 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13547,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":78,"spinlock_wait_cycles":3879491328}
W20260709 13:05:30.529079 21042 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:05:30.531219 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.540618 15235 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 14: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:30.542271 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.555552 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.556510 15235 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:30.516057 22037 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.561306 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.561951 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.562333 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.563092 15228 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:30.568123 21833 consensus_queue.cc:1059] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:30.568486 21942 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:30.570029 21942 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:30.570262 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.570475 21942 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:30.571311 21942 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.572682 21942 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.573055 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.502111 22178 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:30.575798 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:30.479581 22101 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:05:30.580952 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:30.576512 15810 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:30.582396 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.581930 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.583379 15842 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:30.538604 22169 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:30.586737 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.592895 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:30.594012 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:30.595209 22131 raft_consensus.cc:2764] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:05:30.596195 22131 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:30.596302 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:30.612221 21881 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:30.613960 21881 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.566366 22087 raft_consensus.cc:3055] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:30.605959 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.565917 21973 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:30.631947 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.575863 22161 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:30.643716 15228 raft_consensus.cc:2468] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:30.636024 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.635993 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.647315 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.648366 15842 raft_consensus.cc:2393] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:30.650143 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:30.576217 22116 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 13:05:30.656252 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63527.2us
I20260709 13:05:30.625670 21869 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:30.657155 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:30.631517 22087 raft_consensus.cc:740] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:30.581319 21927 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 13:05:30.663100 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 376511us
W20260709 13:05:30.664173 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 272133us
I20260709 13:05:30.637867 22122 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:30.669088 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.632767 22131 raft_consensus.cc:2764] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
W20260709 13:05:30.670647 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 150411us
I20260709 13:05:30.671628 15228 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:30.688652 21927 consensus_queue.cc:1243] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:30.689864 22138 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.672138 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.690850 15229 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:30.691882 22098 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:30.680992 15810 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:30.692938 22098 consensus_queue.cc:1243] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:30.693819 21927 raft_consensus.cc:2804] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:30.694435 21927 raft_consensus.cc:697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:30.690899 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:30.695546 21927 consensus_queue.cc:237] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.696230 22102 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:30.696867 22102 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:30.697283 22102 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:30.646812 22183 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:30.701192 22183 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.701627 22183 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.701964 22105 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:30.649430 21881 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:30.684938 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1643) took 1035 ms. Trace:
I20260709 13:05:30.707269 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.716208 15230 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:30.696331 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:30.591435 22169 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:30.662822 22159 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:30.584688 22178 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.705786 15163 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:30.727283 15236 raft_consensus.cc:1217] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Deduplicated request from leader. Original: 13.6->[16.7-16.7]   Dedup: 16.7->[]
I20260709 13:05:30.679422 15235 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:30.660557 21869 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:30.727876 15236 raft_consensus.cc:1240] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:05:30.715293 15841 rpcz_store.cc:276] 0709 13:05:29.649991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.119885 (+469894us) service_pool.cc:224] Handling call
0709 13:05:30.120258 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.120467 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:05:30.402646 (+282179us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.684411 (+281765us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.684478 (+    67us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:05:30.684528 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.684713 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.684875 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22548,"prepare.queue_time_us":464131,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":6390,"raft.queue_time_us":25165,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":533220,"threads_started":2}
I20260709 13:05:30.716713 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.728801 15237 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:30.717432 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:30.717665 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 67612.1us
I20260709 13:05:30.733040 15126 leader_election.cc:400] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15.
I20260709 13:05:30.717682 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.670089 22188 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:30.681492 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:30.707008 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.714617 22061 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:30.729609 21641 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.736977 21973 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.645610 22019 consensus_queue.cc:260] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.739897 22021 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.725509 22101 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:30.743391 22197 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:30.725178 21833 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.655109 22184 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:05:30.638177 22085 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:30.747308 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65923us
I20260709 13:05:30.667716 22090 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 13:05:30.801023 22179 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 13:05:30.802660 22180 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:05:30.803903 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1681) took 1067 ms. Trace:
W20260709 13:05:30.751006 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 63291.2us
I20260709 13:05:30.761022 22193 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:30.795584 21731 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:30.805050 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:30.805372 21731 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:30.807611 15088 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:30.777230 22188 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:30.804354 15840 rpcz_store.cc:276] 0709 13:05:29.736596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.137072 (+400476us) service_pool.cc:224] Handling call
0709 13:05:30.137598 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.138017 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.138046 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.138602 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.374287 (+235685us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:30.374337 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.374416 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.803665 (+429249us) raft_consensus.cc:1685] finished
0709 13:05:30.803693 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.803845 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5513,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":235196,"threads_started":1,"wal-append.queue_time_us":358236}
I20260709 13:05:30.793700 22019 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:30.800287 22186 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:05:30.765435 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.713967 15160 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:30.715106 21881 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:30.705667 22182 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.843066 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:30.728809 22112 consensus_meta.cc:287] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.003s
I20260709 13:05:30.730240 21869 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.718667 22126 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:30.775517 22098 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:30.793205 22101 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:30.772550 22082 consensus_queue.cc:1059] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:05:30.747877 22061 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.746205 22087 consensus_queue.cc:260] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.789279 22107 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.781648 21641 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.822937 15235 raft_consensus.cc:1240] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:30.824564 15228 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:30.840323 15229 raft_consensus.cc:1240] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:30.840613 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.847602 22198 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:30.850028 22021 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.930593 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.931103 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.931861 15235 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:30.932343 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.932722 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.932551 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.933348 15229 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.854068 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.819640 22169 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:30.938321 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.818738 22102 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:30.840579 22184 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.913595 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:30.822306 22090 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:30.819789 22206 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:30.842546 21881 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.934324 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.914001 22061 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:30.952366 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 203618us
I20260709 13:05:30.762571 15237 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
W20260709 13:05:30.953179 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.949520 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.859573 22159 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:05:30.956269 15235 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:30.957165 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.954507 22184 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:30.951954 22064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.008s	sys 0.004s
I20260709 13:05:30.928117 22091 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:30.995528 15229 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:30.961359 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.964641 15154 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:30.965283 15156 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:30.910071 22112 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.958881 15231 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:30.969467 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:30.974898 22096 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.976603 22144 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:31.032363 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:30.977346 22036 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:30.980116 22148 log.cc:927] Time spent T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.339s	user 0.001s	sys 0.001s
I20260709 13:05:30.981251 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:30.949052 22090 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:30.962949 22183 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:30.995638 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:30.960788 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:31.016263 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.018225 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:30.972402 22064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.346s	user 0.009s	sys 0.005s
W20260709 13:05:30.980489 22057 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.003s
I20260709 13:05:30.975518 22102 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:31.058845 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105766us
W20260709 13:05:31.046254 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 310146us
I20260709 13:05:31.068490 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:31.031708 22144 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:31.032981 15086 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:31.033993 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.089044 15154 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:31.059727 22091 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:31.074930 22126 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:30.996816 22101 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.073356 22096 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.955777 22188 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.062629 21973 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:05:31.077903 22064 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.040290 22036 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:31.067480 22155 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:05:31.141398 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:31.125171 22138 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:31.138331 22033 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.001s
I20260709 13:05:31.121210 22091 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.151815 22091 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:31.151448 22036 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.152801 22036 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:31.132124 22064 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.171186 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1600) took 2320 ms. Trace:
I20260709 13:05:31.171394 15154 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:31.026185 22149 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:31.046092 21976 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.000s
I20260709 13:05:31.171590 15846 rpcz_store.cc:276] 0709 13:05:28.845898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.846404 (+   506us) service_pool.cc:224] Handling call
0709 13:05:28.846759 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.869260 (+ 22501us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:29.318390 (+449130us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.465820 (+147430us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.486859 (+ 21039us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.487016 (+   157us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:29.487051 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.487576 (+   525us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.979524 (+1491948us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:31.154063 (+174539us) raft_consensus.cc:1685] finished
0709 13:05:31.165855 (+ 11792us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.166006 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11835,"prepare.queue_time_us":520061,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":125186,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":16401,"spinlock_wait_cycles":54546176,"thread_start_us":524544,"threads_started":2,"wal-append.queue_time_us":250958}
I20260709 13:05:31.146296 21641 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.173607 15086 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
W20260709 13:05:31.177666 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.178051 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:31.169268 21895 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
W20260709 13:05:31.197423 22217 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:05:31.202760 21881 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:31.204370 21942 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.631s	user 0.005s	sys 0.005s
W20260709 13:05:31.206908 22215 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:05:31.181059 21976 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:31.213315 22216 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:05:31.174423 21869 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.204993 15158 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:31.182673 22149 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.217693 15054 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.821s (SENT) [suppressed 24 similar messages]
I20260709 13:05:31.221534 15843 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:31.121999 22090 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.224436 15841 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:31.164309 22033 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.185295 22178 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.192704 22184 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.187743 21895 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.289496 15159 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:31.228802 22112 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.246810 22064 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:31.175778 22144 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:31.335014 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 171483us
I20260709 13:05:31.190346 22211 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:05:31.350150 15839 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:31.337584 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:31.341112 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.321368 22090 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:31.326422 22183 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:31.369158 22204 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 13:05:31.245100 22149 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.369884 22228 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:31.372520 22144 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:31.438156 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100648us
I20260709 13:05:31.439977 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:31.442636 22019 consensus_meta.cc:287] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 13:05:31.445755 21869 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.452268 15232 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:31.410382 22228 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:31.536356 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.559953 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.336700 22225 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.224s
W20260709 13:05:31.569662 22220 log.cc:927] Time spent T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:05:31.574610 22149 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:31.595649 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.576452 21895 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.561414 21881 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.553843 22228 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:31.650599 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 210713us
I20260709 13:05:31.652266 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.653857 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:31.660650 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.046s	sys 0.079s
W20260709 13:05:31.661558 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.046s	sys 0.079s
I20260709 13:05:31.663134 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.663812 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.671531 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.672287 15154 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:31.674250 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.661697 22225 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:31.681136 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.681725 15162 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 12.
I20260709 13:05:31.681829 15160 raft_consensus.cc:2376] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17. Re-sending same reply.
W20260709 13:05:31.682921 21922 consensus_meta.cc:287] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.171s	user 0.012s	sys 0.000s
I20260709 13:05:31.683070 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.684525 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.685215 15160 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:31.689975 15162 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:31.695595 15160 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:31.700830 22033 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.618088 22188 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:31.711180 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:31.715549 22096 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 13:05:31.732749 15154 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:31.733124 15084 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 13:05:31.752597 22036 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.002s
I20260709 13:05:31.763362 22027 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:31.763962 22027 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:31.780745 15128 connection.cc:256] RPC call timeout handler was delayed by -2.1200814969998873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 13:05:31.782495 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.781612 22188 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.761765 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.835107 15203 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Timed out: RequestConsensusVote RPC to 127.13.163.194:38049 timed out after 1.683s (SENT)
W20260709 13:05:31.835553 22238 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:31.835850 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:31.838606 22064 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:31.841995 22061 consensus_meta.cc:287] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.878s	user 0.012s	sys 0.000s
I20260709 13:05:31.842478 22061 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.843418 22027 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.845932 22027 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:31.855377 22236 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:05:31.859964 22027 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.860636 22027 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.861228 15842 raft_consensus.cc:2393] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:31.833611 22188 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.863018 22027 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:31.868779 22139 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:31.875195 22087 raft_consensus.cc:3060] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:31.882906 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.883386 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.885466 22108 raft_consensus.cc:3055] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:31.885742 22096 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:31.885929 22108 raft_consensus.cc:740] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:31.886324 22096 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.886847 22108 consensus_queue.cc:260] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:31.893784 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.170042 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:31.897498 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.903614 14979 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 16, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:31.905769 14979 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.898264 22101 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:31.909015 22019 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.909767 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1617) took 1593 ms. Trace:
I20260709 13:05:31.910336 15159 rpcz_store.cc:276] 0709 13:05:30.315977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.316128 (+   151us) service_pool.cc:224] Handling call
0709 13:05:31.909491 (+1593363us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54001266d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:31.909690 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152566016}
I20260709 13:05:31.911262 22037 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:31.913486 22116 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:31.914042 22116 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.915928 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.916326 22116 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.909624 22019 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.917301 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.917557 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.918546 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.920257 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.921763 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:31.907955 14979 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:31.923606 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:31.912722 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.925450 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:31.927227 15054 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.048s	sys 0.108s
W20260709 13:05:31.927636 15054 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.048s	sys 0.108s
W20260709 13:05:31.928491 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 710780us
W20260709 13:05:31.932206 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.932515 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.935282 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:31.937919 15086 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:31.939095 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.902699 22061 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:31.943867 15086 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:31.948275 15082 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:31.925593 22116 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:31.938558 22036 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.908967 22101 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.954478 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:31.945183 22130 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:31.950011 22112 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.949793 22188 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.953493 22183 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:31.956650 22182 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:31.960172 22096 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.982405 22182 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.983182 22113 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:31.983861 22113 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:31.988940 22183 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:31.995434 15161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2697) took 2061 ms (client timeout 1821 ms). Trace:
W20260709 13:05:31.995886 15161 rpcz_store.cc:269] 0709 13:05:29.934160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.934485 (+   325us) service_pool.cc:224] Handling call
0709 13:05:31.995240 (+2060755us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54000857d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:31.995360 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673219456}
I20260709 13:05:31.963034 22116 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:31.997867 22113 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.998495 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2712) took 1036 ms. Trace:
I20260709 13:05:31.998821 22116 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:31.999322 15081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1637) took 2706 ms (client timeout 2930 ms). Trace:
W20260709 13:05:31.999714 15081 rpcz_store.cc:269] 0709 13:05:29.292682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.295614 (+  2932us) service_pool.cc:224] Handling call
0709 13:05:31.999136 (+2703522us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b5400118854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:31.999268 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497196416}
I20260709 13:05:31.998874 15158 rpcz_store.cc:276] 0709 13:05:30.961631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.961774 (+   143us) service_pool.cc:224] Handling call
0709 13:05:30.974439 (+ 12665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.240966 (+266527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.998263 (+757297us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540013f6d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:31.998439 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529714048}
I20260709 13:05:32.002441 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1621) took 1427 ms. Trace:
I20260709 13:05:32.002961 15839 rpcz_store.cc:276] 0709 13:05:30.575305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.744983 (+169678us) service_pool.cc:224] Handling call
0709 13:05:30.745313 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.350795 (+605482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.002131 (+651336us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.002366 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390413696}
I20260709 13:05:32.003834 22228 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.004474 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:31.970690 22133 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:32.006107 15839 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:32.009496 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:32.010342 15156 raft_consensus.cc:2368] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19.
I20260709 13:05:31.974957 22027 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.010174 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:32.013139 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 230149us
W20260709 13:05:32.014750 15128 connection.cc:256] RPC call timeout handler was delayed by -1.9640572629999724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
W20260709 13:05:32.015463 15128 connection.cc:256] RPC call timeout handler was delayed by -1.7069814969999015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260709 13:05:32.016206 15128 connection.cc:256] RPC call timeout handler was delayed by -1.5467814969999836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2383s for a response
W20260709 13:05:32.016677 15128 connection.cc:256] RPC call timeout handler was delayed by -1.2740938450000385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
W20260709 13:05:32.017462 15128 connection.cc:256] RPC call timeout handler was delayed by -1.2067098339998665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
I20260709 13:05:32.017211 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.019121 15158 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:32.020340 15162 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 17. Ops: []
I20260709 13:05:31.967521 22036 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:31.978077 22130 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.009059 22133 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.000003 22113 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:31.980309 22112 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.012953 22092 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:32.045011 22092 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.039292 22133 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:32.062453 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.063146 15081 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:05:32.063231 15080 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:32.050843 15233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1509) took 1999 ms (client timeout 1864 ms). Trace:
I20260709 13:05:32.064464 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:32.064596 15233 rpcz_store.cc:269] 0709 13:05:30.051465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.052074 (+   609us) service_pool.cc:224] Handling call
0709 13:05:32.050609 (+1998535us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b54000591d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.050760 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89164544}
I20260709 13:05:32.065419 15083 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:32.052488 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.014237 22182 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.043627 22212 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:32.011528 22188 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:05:32.044473 15079 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:32.065418 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.097550 15081 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 23.
I20260709 13:05:32.023185 21942 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.041983 15128 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.071695 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:32.104586 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 63160.3us
I20260709 13:05:32.106958 15128 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:32.044263 22019 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:32.157684 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 320104us
W20260709 13:05:32.158165 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51339.8us
I20260709 13:05:32.050211 22130 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.177211 15839 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:32.018625 22027 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:32.159387 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58604.1us
I20260709 13:05:32.175812 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.192162 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.099570 22182 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.203413 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.103145 21895 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.160876 21942 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.205579 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:32.212318 22245 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:05:32.215658 22101 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.216331 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:32.201800 22243 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:05:32.217267 15086 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:32.217698 22112 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:32.205629 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.218225 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.219115 15081 raft_consensus.cc:2393] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:32.180230 22084 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.219671 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.221022 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.160962 22166 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:32.253408 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47924.2us
I20260709 13:05:32.262900 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1526) took 1089 ms. Trace:
W20260709 13:05:32.262101 15051 connection.cc:256] RPC call timeout handler was delayed by -1.4201933190000773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:05:32.264992 15051 connection.cc:256] RPC call timeout handler was delayed by -1.2761933190000718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 13:05:32.257357 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56355.5us
W20260709 13:05:32.268043 15051 connection.cc:256] RPC call timeout handler was delayed by -1.2176933190000909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:05:32.243467 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64668us
I20260709 13:05:32.265872 15203 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:32.263511 15232 rpcz_store.cc:276] 0709 13:05:31.173500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.173671 (+   171us) service_pool.cc:224] Handling call
0709 13:05:31.186801 (+ 13130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.452924 (+266123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.262585 (+809661us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540005b754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.262827 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578359168}
W20260709 13:05:32.258220 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 308478us
I20260709 13:05:32.175400 22235 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 13:05:32.247184 22153 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 2.598s	user 0.003s	sys 0.008s
I20260709 13:05:32.161098 22068 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:32.275084 15051 connection.cc:256] RPC call timeout handler was delayed by -1.1582933190001086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 13:05:32.276714 15051 connection.cc:256] RPC call timeout handler was delayed by -1.1150933190000387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:05:32.277397 15051 connection.cc:256] RPC call timeout handler was delayed by -1.0619933190000665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:05:32.277760 15051 connection.cc:256] RPC call timeout handler was delayed by -1.0547933190000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
I20260709 13:05:32.249161 15086 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:32.278012 15079 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:32.275830 15128 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.204717 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1626) took 1056 ms. Trace:
I20260709 13:05:32.202754 21929 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:32.218353 22112 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.266716 22197 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:32.258324 22103 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:32.220603 22182 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.205181 22064 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:32.281529 15839 rpcz_store.cc:276] 0709 13:05:31.148308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.202916 (+1054608us) service_pool.cc:224] Handling call
0709 13:05:32.204435 (+  1519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.297086 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:32.282838 22203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.310s	user 0.011s	sys 0.011s
I20260709 13:05:32.250080 22144 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:05:32.248643 21895 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.309996 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.273988 22106 consensus_queue.cc:1059] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:05:32.287974 22019 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.257872 22084 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.277124 22193 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 13:05:32.305568 22203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.012s	sys 0.011s
I20260709 13:05:32.282907 22212 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:05:32.309401 22197 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:32.317381 22064 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:32.316040 22200 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:05:32.351002 22228 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:32.352373 22146 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 2.847s	user 0.010s	sys 0.010s
I20260709 13:05:32.352968 22084 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.310567 22068 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.355002 22092 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.290192 22112 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.299924 22103 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.352180 22182 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.282322 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.417971 15160 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:32.418614 15081 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:32.282220 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.416216 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1610) took 3172 ms. Trace:
W20260709 13:05:32.425762 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 158486us
W20260709 13:05:32.429500 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 163638us
I20260709 13:05:32.430820 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:32.433982 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.417999 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.438421 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.440004 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:32.440693 21929 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.306787 21942 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.425053 15847 rpcz_store.cc:276] 0709 13:05:29.243572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.243741 (+   169us) service_pool.cc:224] Handling call
0709 13:05:29.244137 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.250837 (+  6700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.250881 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.281096 (+ 30215us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.368797 (+ 87701us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:29.368875 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.368981 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.351791 (+2982810us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:32.415271 (+ 63480us) raft_consensus.cc:1685] finished
0709 13:05:32.415326 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.415825 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6198,"raft.queue_time_us":11391,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":22628,"spinlock_wait_cycles":12548864,"thread_start_us":39024,"threads_started":1,"wal-append.queue_time_us":134366}
I20260709 13:05:32.334533 22036 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:32.450109 22044 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:32.428598 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:32.375174 22197 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.473429 22087 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.473850 21895 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.474339 22087 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.479235 22250 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:32.334311 22019 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.480010 22250 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:32.481331 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.482774 22019 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.485023 22087 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.485483 21895 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:32.420536 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.487586 15081 raft_consensus.cc:2376] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18. Re-sending same reply.
W20260709 13:05:32.486722 22037 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.005s
I20260709 13:05:32.490363 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.491917 22112 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.446604 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.492486 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:32.502126 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 71604.8us
I20260709 13:05:32.514030 15848 raft_consensus.cc:2410] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:05:32.515026 22037 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.515795 22103 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.515852 22037 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.517683 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.519213 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.519510 22184 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.520195 22184 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.494735 22064 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:32.522027 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1512) took 2453 ms. Trace:
I20260709 13:05:32.522490 15234 rpcz_store.cc:276] 0709 13:05:30.068107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.068324 (+   217us) service_pool.cc:224] Handling call
0709 13:05:30.068642 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.320689 (+252047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.521275 (+2200586us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400276fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.521696 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432780544}
I20260709 13:05:32.523551 22250 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.526422 15236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2456) took 1845 ms. Trace:
I20260709 13:05:32.526829 15236 rpcz_store.cc:276] 0709 13:05:30.681089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.681218 (+   129us) service_pool.cc:224] Handling call
0709 13:05:30.719450 (+ 38232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.728343 (+  8893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.525850 (+1797507us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b5400175f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.526217 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17713536}
I20260709 13:05:32.518982 22037 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:32.531962 15843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1647) took 2328 ms (client timeout 1612 ms). Trace:
W20260709 13:05:32.533691 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 105383us
I20260709 13:05:32.537896 22261 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:32.538637 22261 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:32.539232 22261 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:32.546285 21833 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:32.546952 21833 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:32.552002 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.552618 21833 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.554239 21833 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:32.554405 21731 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.555619 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.556129 21731 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:32.550279 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.558202 15160 raft_consensus.cc:2410] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:32.533703 15840 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:32.561151 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1622) took 1971 ms. Trace:
I20260709 13:05:32.561585 22184 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.561517 15840 rpcz_store.cc:276] 0709 13:05:30.589901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.825348 (+235447us) service_pool.cc:224] Handling call
0709 13:05:32.533017 (+1707669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.559470 (+ 26453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.560896 (+  1426us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54001e0fd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.561076 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3789688704}
I20260709 13:05:32.562942 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:32.563604 15840 raft_consensus.cc:2376] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18. Re-sending same reply.
I20260709 13:05:32.566852 22108 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:32.560832 21930 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.550315 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:32.572021 15846 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:32.572343 22118 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:32.573164 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:32.573364 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.573894 15846 raft_consensus.cc:2368] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19.
I20260709 13:05:32.574223 15847 raft_consensus.cc:2410] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:05:32.574739 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.575505 15846 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:32.575933 15847 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:32.576432 22183 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.554396 22027 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:32.577062 22133 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.005s
I20260709 13:05:32.585821 22133 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.581470 22149 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.587643 22222 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:32.587483 22149 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.588315 22222 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:32.522611 22064 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.589987 22222 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:32.581633 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 273374us
I20260709 13:05:32.591813 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.593175 22230 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:32.593695 22230 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:32.593827 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.595078 22230 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.597164 22027 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:32.597965 22027 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:32.598762 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.599267 22027 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.582733 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.602363 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:32.582604 22198 raft_consensus.cc:3055] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:32.605340 22198 raft_consensus.cc:740] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:32.605695 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3190) took 2210 ms. Trace:
I20260709 13:05:32.606185 15088 rpcz_store.cc:276] 0709 13:05:30.395469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.395637 (+   168us) service_pool.cc:224] Handling call
0709 13:05:30.420021 (+ 24384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.813603 (+393582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.605169 (+1791566us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540014da54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.605509 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890810624}
I20260709 13:05:32.590495 22149 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:32.595736 22184 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:32.584782 22203 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:32.576578 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1624) took 1827 ms. Trace:
I20260709 13:05:32.520663 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.591940 22222 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:32.608939 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.613638 15847 rpcz_store.cc:276] 0709 13:05:30.749322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.575027 (+1825705us) service_pool.cc:224] Handling call
0709 13:05:32.575491 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.576324 (+   833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.576500 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.566931 15234 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:32.605039 21930 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:32.532636 15843 rpcz_store.cc:269] 0709 13:05:30.203578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.228162 (+ 24584us) service_pool.cc:224] Handling call
0709 13:05:32.531423 (+2303261us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54001e0fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.531688 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2028276608}
I20260709 13:05:32.606895 22198 consensus_queue.cc:260] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.607376 22133 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.613070 22203 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:32.635542 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:32.641959 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.640798 15840 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:32.646004 15813 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.634325 22184 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:32.651453 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.650128 15082 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:32.652868 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48014.2us
I20260709 13:05:32.656442 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.659895 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.660974 22183 raft_consensus.cc:2804] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:32.660801 22203 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.661525 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.661692 22183 raft_consensus.cc:697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:32.663003 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.663790 15236 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:32.663280 22183 consensus_queue.cc:237] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.658437 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.662804 22203 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:32.664608 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.675138 15081 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:32.665231 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.676034 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:32.681885 15843 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:32.681326 15082 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:32.686326 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1625) took 1640 ms. Trace:
I20260709 13:05:32.686858 15843 rpcz_store.cc:276] 0709 13:05:31.042323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.650284 (+1607961us) service_pool.cc:224] Handling call
0709 13:05:32.675885 (+ 25601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.682345 (+  6460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.682974 (+   629us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.676275 22240 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.697s	user 0.005s	sys 0.006s
I20260709 13:05:32.679864 22230 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:32.694809 15847 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:05:32.700210 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.702879 15081 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:32.703243 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.704430 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.690757 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.707243 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:32.709672 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.710558 15088 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:32.711902 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:32.704793 22198 raft_consensus.cc:3060] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:32.725783 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:32.698362 22149 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:05:32.737067 15088 raft_consensus.cc:2410] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:32.737237 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:32.743078 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 63618us
I20260709 13:05:32.746519 15081 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:32.748203 15843 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:32.749624 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:32.749851 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.750435 15843 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:32.750707 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.748208 15081 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:05:32.753134 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.753708 15083 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:32.754408 15081 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:32.755843 22227 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.756608 22227 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:32.760969 22266 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:05:32.767373 22139 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.004s
I20260709 13:05:32.768204 22149 consensus_queue.cc:1243] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 12daa67457954a0cab27e32bf85fa115 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:32.768798 22149 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:32.769155 22149 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:32.770272 22149 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.771704 22149 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:32.773028 21833 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:32.773641 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92898us
I20260709 13:05:32.773918 21833 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.774896 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.776521 21833 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.778961 15847 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:05:32.756722 15083 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:05:32.783114 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.789765 15842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1693) took 1835 ms (client timeout 2141 ms). Trace:
W20260709 13:05:32.790355 15842 rpcz_store.cc:269] 0709 13:05:30.953924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.995306 (+ 41382us) service_pool.cc:224] Handling call
0709 13:05:32.789241 (+1793935us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b5400806454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.789537 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884120960}
I20260709 13:05:32.791800 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:32.793200 22184 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:32.794432 22270 log.cc:927] Time spent T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:05:32.795776 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 157395us
I20260709 13:05:32.796171 15841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1620) took 2230 ms. Trace:
I20260709 13:05:32.796590 15841 rpcz_store.cc:276] 0709 13:05:30.565315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.728776 (+163461us) service_pool.cc:224] Handling call
0709 13:05:30.729057 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.233256 (+504199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.795704 (+1562448us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540074d454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.796002 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138816896}
I20260709 13:05:32.798678 15841 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:32.799592 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:32.801079 22029 raft_consensus.cc:3055] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:32.801635 22029 raft_consensus.cc:740] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:32.802997 22029 consensus_queue.cc:260] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.804347 22029 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:32.805748 15839 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:32.819535 15842 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:32.831347 15082 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:32.835305 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.759476 22227 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:32.836704 15052 leader_election.cc:400] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19.
W20260709 13:05:32.838419 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.839246 15052 leader_election.cc:403] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:05:32.839965 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.783756 22139 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:32.738952 22261 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:32.850301 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 105560us
I20260709 13:05:32.842909 22133 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:32.861083 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.861450 22130 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.865578 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66114.6us
I20260709 13:05:32.843521 22139 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:32.870266 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2707) took 2047 ms. Trace:
W20260709 13:05:32.874197 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3178) took 3260 ms (client timeout 2246 ms). Trace:
I20260709 13:05:32.871881 15845 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1676) took 3137 ms. Trace:
I20260709 13:05:32.893949 15233 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:32.901561 15840 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:05:32.845687 22250 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:32.870520 15228 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:32.907866 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2718) took 1633 ms. Trace:
I20260709 13:05:32.908161 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1631) took 2180 ms. Trace:
W20260709 13:05:32.908298 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 147653us
I20260709 13:05:32.910422 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:32.871855 21942 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:05:32.918700 22131 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.919975 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.920568 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 143743us
I20260709 13:05:32.923322 21931 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.925037 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:32.925431 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.925135 15844 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:05:32.930930 22267 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:05:32.872202 22261 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:32.909230 22258 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.937011 15845 rpcz_store.cc:276] 0709 13:05:29.734710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.122059 (+387349us) service_pool.cc:224] Handling call
0709 13:05:30.122387 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.191476 (+ 69089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.191510 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.207432 (+ 15922us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.654881 (+447449us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:30.686242 (+ 31361us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.686599 (+   357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.871241 (+2184642us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b540074e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.871308 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.871391 (+    83us) raft_consensus.cc:1685] finished
0709 13:05:32.871475 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.871701 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36999,"prepare.queue_time_us":14097,"prepare.run_cpu_time_us":2752,"prepare.run_wall_time_us":10241,"raft.queue_time_us":27863,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":6967,"spinlock_wait_cycles":97306880,"thread_start_us":376546,"threads_started":1,"wal-append.queue_time_us":482956,"wal-append.run_cpu_time_us":2732,"wal-append.run_wall_time_us":1023305}
I20260709 13:05:32.938850 15155 rpcz_store.cc:276] 0709 13:05:31.217939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.218155 (+   216us) service_pool.cc:224] Handling call
0709 13:05:31.224961 (+  6806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:31.249823 (+ 24862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.249871 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.256515 (+  6644us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.732629 (+476114us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:31.732689 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.732786 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.599846 (+867060us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:32.806631 (+206785us) raft_consensus.cc:1685] finished
0709 13:05:32.806661 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.851587 (+ 44926us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19431,"raft.queue_time_us":145628,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":51006,"spinlock_wait_cycles":13080448,"thread_start_us":475618,"threads_started":1,"wal-append.queue_time_us":646491}
I20260709 13:05:32.944095 22109 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.962703 22109 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:32.936766 15079 rpcz_store.cc:269] 0709 13:05:29.613740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.613879 (+   139us) service_pool.cc:224] Handling call
0709 13:05:32.873755 (+3259876us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.873965 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5582517120}
I20260709 13:05:32.894891 22139 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:32.965096 22033 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:32.965546 22033 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:32.965965 22033 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:32.969769 15848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1672) took 3236 ms (client timeout 2621 ms). Trace:
W20260709 13:05:32.970283 15848 rpcz_store.cc:269] 0709 13:05:29.733557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.077665 (+344108us) service_pool.cc:224] Handling call
0709 13:05:32.969463 (+2891798us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b54005a3a54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:32.969691 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5552489216}
I20260709 13:05:32.938962 15163 rpcz_store.cc:276] 0709 13:05:30.727539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.728128 (+   589us) service_pool.cc:224] Handling call
0709 13:05:30.728399 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.812470 (+ 84071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.812507 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.812556 (+    49us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:30.812598 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.907659 (+2095061us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b540033f754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:32.907739 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.907975 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192867712}
W20260709 13:05:32.954404 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 74648.9us
W20260709 13:05:32.972520 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.972877 22131 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.973572 15841 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:32.975342 15841 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:32.975520 22131 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.975831 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.976568 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.978387 15161 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:32.979291 21931 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.980219 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.980669 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:32.981199 22133 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:32.981884 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.982875 22133 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:32.983425 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:32.906895 22130 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:32.985530 22250 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:32.985962 22087 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:32.987069 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89104.1us
I20260709 13:05:32.986933 22250 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.987943 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:32.993628 21895 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.994220 21895 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.996299 21895 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:32.997089 22027 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:32.897768 22227 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:05:32.999517 22189 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:32.999967 22189 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.000334 22189 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:32.979324 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.001331 22227 consensus_queue.cc:1059] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:33.002249 22227 consensus_queue.cc:1243] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:33.002735 22227 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.003417 21042 raft_consensus.cc:3055] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:33.003954 21042 raft_consensus.cc:740] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:33.006883 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.008054 15158 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 16: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:33.008745 15086 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:33.009588 22101 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.009893 22230 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:33.010150 22101 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:33.011633 15228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2721) took 2147 ms (client timeout 1780 ms). Trace:
W20260709 13:05:33.012159 15228 rpcz_store.cc:269] 0709 13:05:30.863918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.931998 (+ 68080us) service_pool.cc:224] Handling call
0709 13:05:33.011350 (+2079352us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400294d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:33.011535 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4455134464}
W20260709 13:05:33.014130 15084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1648) took 2831 ms (client timeout 1934 ms). Trace:
W20260709 13:05:33.014240 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 130417us
I20260709 13:05:33.003284 22227 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:32.976832 15841 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:33.014927 22118 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:33.014407 15087 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:05:33.027417 15087 pending_rounds.cc:85] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 12
I20260709 13:05:33.027297 15843 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
W20260709 13:05:33.016799 15084 rpcz_store.cc:269] 0709 13:05:30.182279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.183116 (+   837us) service_pool.cc:224] Handling call
0709 13:05:33.013566 (+2830450us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540011a3d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:33.013866 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3556485376}
W20260709 13:05:33.034554 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47615.6us
I20260709 13:05:33.047708 22130 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:33.049914 22109 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.049898 15081 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:05:33.027845 15087 pending_rounds.cc:107] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:05:33.050657 15848 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:33.051029 15087 raft_consensus.cc:2889] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:33.051950 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.052623 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.053010 15846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1692) took 2228 ms. Trace:
I20260709 13:05:33.053448 22249 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:33.053392 15846 rpcz_store.cc:276] 0709 13:05:30.824267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.576202 (+1751935us) service_pool.cc:224] Handling call
0709 13:05:32.576650 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.643360 (+ 66710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.643427 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.673587 (+ 30160us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.849701 (+176114us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:32.849739 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.849800 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.052782 (+202982us) raft_consensus.cc:1685] finished
0709 13:05:33.052800 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.052948 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":140199,"threads_started":1,"wal-append.queue_time_us":358088}
I20260709 13:05:33.054339 22249 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.055603 22261 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.055940 15231 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:05:33.056329 22261 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.057216 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.057358 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.058852 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.059784 15234 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:33.059894 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:33.053046 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.060949 22264 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.061371 22269 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.289s	user 0.000s	sys 0.003s
I20260709 13:05:33.061517 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.061764 22264 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.062724 21929 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.063303 21929 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.063771 22117 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.015749 21931 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:33.066627 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1623) took 2360 ms. Trace:
I20260709 13:05:33.066900 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.067137 15839 rpcz_store.cc:276] 0709 13:05:30.706108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.307196 (+1601088us) service_pool.cc:224] Handling call
0709 13:05:32.805284 (+498088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.806256 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.066365 (+260109us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b5400806450. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:33.066536 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144373632}
I20260709 13:05:33.068405 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:33.069461 22271 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.341s	user 0.000s	sys 0.002s
I20260709 13:05:33.069890 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:32.997179 22036 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:05:33.050573 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1694) took 1986 ms. Trace:
I20260709 13:05:33.058997 22261 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.050740 15081 pending_rounds.cc:85] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 9
I20260709 13:05:33.074054 15840 rpcz_store.cc:276] 0709 13:05:31.063863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.665764 (+1601901us) service_pool.cc:224] Handling call
0709 13:05:32.694470 (+ 28706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.946022 (+251552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.050249 (+104227us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54007c4554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:33.050469 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437468032}
I20260709 13:05:33.077230 15081 pending_rounds.cc:107] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:05:33.075359 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.077574 15081 raft_consensus.cc:2889] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.064517 22264 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.064302 22117 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.078012 15844 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:33.076421 22133 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:33.080796 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:33.031445 22118 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:33.061836 15201 connection.cc:256] RPC call timeout handler was delayed by -1.3583850129999746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2621s for a response
I20260709 13:05:32.939169 15230 rpcz_store.cc:276] 0709 13:05:30.822297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.823694 (+  1397us) service_pool.cc:224] Handling call
0709 13:05:30.849254 (+ 25560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.849630 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:30.849967 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.865780 (+1015813us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003e3620. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f58423a
0709 13:05:32.268728 (+402948us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.586238 (+317510us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.587435 (+  1197us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.588194 (+   759us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:32.588650 (+   456us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.609127 (+ 20477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.632238 (+ 23111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.848678 (+216440us) raft_consensus.cc:1685] finished
0709 13:05:32.848699 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.870080 (+ 21381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45586,"prepare.queue_time_us":175860,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":127,"raft.queue_time_us":146157,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":5597,"spinlock_wait_cycles":12232832,"thread_start_us":790653,"threads_started":3,"wal-append.queue_time_us":555333,"wal-append.run_cpu_time_us":2077,"wal-append.run_wall_time_us":1187539}
W20260709 13:05:33.101640 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.100258 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.105931 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.115837 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.115952 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.117997 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.092741 22133 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:33.115367 21042 consensus_queue.cc:260] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.102659 22118 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:33.145938 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.146230 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.080119 22222 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:05:33.152920 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 168180us
I20260709 13:05:33.146906 15160 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:33.160372 15232 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:33.160912 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.126281 22249 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.167500 15840 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:33.173792 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.173494 22131 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:33.174420 22183 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:33.179642 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.185518 15234 raft_consensus.cc:2410] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:05:33.179911 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.183459 15157 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:33.148831 22133 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:33.180015 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.148236 22203 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.180830 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.115499 22130 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:05:33.188686 22183 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.189517 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.201272 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.225231 15814 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:33.226588 15235 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:33.232609 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72967.9us
I20260709 13:05:33.233139 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:33.233532 15080 consensus_meta.cc:287] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.015s	user 0.011s	sys 0.011s
I20260709 13:05:33.234048 15080 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:33.234416 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.234772 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1655) took 1172 ms. Trace:
I20260709 13:05:33.234820 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.235229 15080 rpcz_store.cc:276] 0709 13:05:32.062084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.062233 (+   149us) service_pool.cc:224] Handling call
0709 13:05:33.234656 (+1172423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:33.235445 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.236970 15231 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:33.235745 15234 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:33.238029 22117 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.238435 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.239301 15229 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
I20260709 13:05:33.157028 22264 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.240942 22041 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:33.241421 22264 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:33.243964 22264 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:33.125995 22230 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:05:33.195202 21942 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:33.245257 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.245352 21942 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.246369 15237 raft_consensus.cc:2410] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:05:33.247623 22183 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:33.247915 21942 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:33.248487 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.249531 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.251431 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.253057 22029 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:33.253540 22029 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:33.254685 22029 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.256230 22029 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:33.228076 21929 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.257441 21042 raft_consensus.cc:3060] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:33.258004 21930 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19.
I20260709 13:05:33.166292 22139 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:33.224349 22087 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.262797 22087 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.263484 22189 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:33.264647 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 186361us
I20260709 13:05:33.263509 22139 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:33.265904 15840 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:33.267206 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.268345 15840 raft_consensus.cc:2410] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:05:33.275620 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.231161 15155 pending_rounds.cc:85] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 6
I20260709 13:05:33.277072 15155 pending_rounds.cc:107] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 16.7
I20260709 13:05:33.277427 15155 raft_consensus.cc:2889] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.252516 22230 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:05:33.286291 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 61146.8us
I20260709 13:05:33.293984 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:33.306545 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.291348 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:33.298458 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40386 (request call id 3203) took 1224 ms. Trace:
I20260709 13:05:33.300896 15084 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:33.300998 22046 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:33.301468 22061 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:33.301703 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.301811 22189 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.302376 15840 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:33.305347 15081 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:33.305955 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.321408 15237 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 24: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:33.331431 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.307965 15086 rpcz_store.cc:276] 0709 13:05:32.065189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.281951 (+216762us) service_pool.cc:224] Handling call
0709 13:05:33.284697 (+1002746us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540011a8d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:33.289342 (+  4645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354706688}
I20260709 13:05:33.284765 21931 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.341254 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.343871 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1659) took 1394 ms. Trace:
I20260709 13:05:33.346375 15082 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:33.295684 22183 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:33.353869 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.350299 15847 rpcz_store.cc:276] 0709 13:05:31.949169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.805839 (+856670us) service_pool.cc:224] Handling call
0709 13:05:32.806412 (+   573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:32.851845 (+ 45433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.851882 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:32.902358 (+ 50476us) log.cc:844] Serialized 52 byte log entry
0709 13:05:33.081805 (+179447us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:33.081852 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.082016 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.343628 (+261612us) raft_consensus.cc:1685] finished
0709 13:05:33.343650 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.343812 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5955,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8207,"thread_start_us":119873,"threads_started":1,"wal-append.queue_time_us":387002}
I20260709 13:05:33.290072 22230 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:33.361670 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.362253 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.353277 15233 raft_consensus.cc:1240] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:33.354753 15234 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:33.355314 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.363024 15230 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:33.373915 15841 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:33.373863 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 263919us
I20260709 13:05:33.311558 22046 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:33.389701 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 138436us
I20260709 13:05:33.385645 15231 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:33.347540 21895 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:33.346902 22249 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:33.392884 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.394332 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.395252 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.404017 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.405238 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.409202 15229 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:33.410135 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.411898 15232 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:33.425689 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.433769 15840 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
W20260709 13:05:33.422013 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85584.6us
I20260709 13:05:33.418817 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.435528 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.421118 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.422173 15080 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:33.437561 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.437770 22227 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.437896 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.439218 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:33.439790 15840 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:33.439473 22033 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:33.440162 15843 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:33.418474 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:33.441069 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 217233us
I20260709 13:05:33.421705 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.421797 15084 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:33.445112 15842 raft_consensus.cc:2410] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:33.402874 22046 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:33.445719 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 200031us
I20260709 13:05:33.405854 21895 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.442015 15231 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
I20260709 13:05:33.443480 15847 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:33.456387 15843 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 20. Ops: [18.14-18.14]
I20260709 13:05:33.454155 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:33.463330 15810 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.467448 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.469877 15081 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:33.471138 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.468715 15233 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:33.474577 15230 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:33.382586 22203 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.477137 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.455555 21895 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:33.481770 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.405364 21931 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.483759 15160 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:33.484133 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.455530 22227 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.494179 15229 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:33.491865 15086 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:33.425046 22037 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:05:33.504655 15237 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:33.452382 22133 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.501335 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:33.517089 15082 raft_consensus.cc:2410] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:33.477695 22184 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.477558 22261 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:33.513774 15842 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:33.519609 15843 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:05:33.522861 15843 pending_rounds.cc:85] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 12
I20260709 13:05:33.523393 15843 pending_rounds.cc:107] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:05:33.526113 15843 raft_consensus.cc:2889] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.529438 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:33.531076 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 43683.5us
I20260709 13:05:33.520345 22189 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:05:33.533407 15159 raft_consensus.cc:1275] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 13:05:33.534870 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 84143.1us
I20260709 13:05:33.537045 22087 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:33.537806 22280 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:05:33.538275 21931 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:33.539592 22182 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:33.540050 22182 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.540375 22182 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:33.530843 22130 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:33.542186 22130 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:33.543243 22130 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:33.548173 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 73804.7us
W20260709 13:05:33.549765 22285 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:05:33.551122 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.554546 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.555680 22264 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:33.558699 15203 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.560598 22290 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:33.560788 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.561739 22059 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:33.562497 22059 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:33.562903 22133 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.564030 15079 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:33.567203 22033 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:33.568786 21948 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:33.569223 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.570478 15842 raft_consensus.cc:1240] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:33.576460 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 236205us
I20260709 13:05:33.577453 15234 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:33.562122 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.581650 15840 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:33.585645 15201 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:33.581928 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45947.7us
I20260709 13:05:33.588331 22059 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:33.588393 15128 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.565s (SENT)
I20260709 13:05:33.520605 22041 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:05:33.593624 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.593855 22189 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:33.594916 22189 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:33.596112 22189 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:33.597769 22189 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:33.603366 22203 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:33.582861 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 59087.6us
I20260709 13:05:33.521852 22261 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:33.616142 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54082.5us
I20260709 13:05:33.621692 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.631160 22041 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:33.631976 22041 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.642652 22041 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.643666 22200 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:33.642681 15840 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:33.645035 15840 pending_rounds.cc:85] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 11
I20260709 13:05:33.646978 15840 pending_rounds.cc:107] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:33.649214 15840 raft_consensus.cc:2889] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.647217 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.649147 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.648180 22203 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:33.648330 22260 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:33.649137 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:33.647838 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.647032 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:33.581437 22139 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:33.663141 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.646001 22046 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.670159 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:33.670864 15234 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
W20260709 13:05:33.653294 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 100550us
I20260709 13:05:33.512573 22227 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:33.659922 15080 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 24.
W20260709 13:05:33.672925 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 87421us
I20260709 13:05:33.663166 15156 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 14.
I20260709 13:05:33.665622 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:33.633116 22261 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:33.700013 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:33.701061 15158 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:33.696885 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.648156 21948 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:33.704233 15848 raft_consensus.cc:2368] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
W20260709 13:05:33.691974 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122852us
I20260709 13:05:33.697988 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.706688 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:33.707288 15234 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:33.528177 22230 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.709100 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:33.669697 22139 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:33.702247 15080 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
W20260709 13:05:33.709964 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.709530 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 212975us
I20260709 13:05:33.710495 15086 raft_consensus.cc:1275] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:05:33.717823 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.718540 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:33.719681 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.720376 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.722241 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.719201 15086 pending_rounds.cc:85] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 10
I20260709 13:05:33.723930 15086 pending_rounds.cc:107] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:05:33.724205 15086 raft_consensus.cc:2889] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.726701 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.724821 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.729517 15156 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:33.732163 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.734994 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.735695 15085 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:33.744051 15081 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:33.743304 15084 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:33.701790 22203 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.750332 15083 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:33.748310 21948 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:05:33.751364 15810 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.753000 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.754429 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.755276 22037 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.755837 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.756068 22037 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.757090 22037 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:33.757630 22263 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:33.757952 22037 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:33.758066 22263 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.758580 22263 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:33.762115 22118 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:05:33.762799 22118 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.763028 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:33.724713 22264 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:33.757139 15128 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.741s (SENT)
I20260709 13:05:33.762003 22046 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:33.701411 22112 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:05:33.778264 15840 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:33.749269 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:33.773088 22061 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:33.747840 15234 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:33.799129 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49934.2us
I20260709 13:05:33.803840 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:33.797978 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:33.808451 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:33.808866 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53133.7us
I20260709 13:05:33.765033 22222 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:05:33.777577 22264 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:33.817680 22041 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:05:33.830332 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:33.845890 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.826145 21042 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:05:33.838075 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:33.825671 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 68582.9us
I20260709 13:05:33.849356 15085 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:33.847227 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:33.869385 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.826013 21833 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:33.825675 22203 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.824221 22182 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.856823 22033 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:33.881032 15128 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.744s (SENT)
I20260709 13:05:33.881596 22033 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.881726 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:33.825850 22289 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:33.883049 22289 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.883526 22289 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:33.883695 15128 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.772s (SENT)
I20260709 13:05:33.884277 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:33.882191 22033 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:33.850461 22139 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.885468 22249 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:33.886112 22139 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:33.834014 22287 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:33.887929 22287 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.831029 22261 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.860931 22264 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:33.890024 22261 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:33.888397 22203 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.887084 21895 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:33.899044 15840 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:05:33.881733 22183 consensus_meta.cc:287] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.002s
I20260709 13:05:33.821098 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:33.847093 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2520) took 3630 ms. Trace:
I20260709 13:05:33.902392 15087 rpcz_store.cc:276] 0709 13:05:30.217173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.274933 (+ 57760us) service_pool.cc:224] Handling call
0709 13:05:33.013828 (+2738895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.051781 (+ 37953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.846838 (+795057us) spinlock_profiling.cc:243] Waited 2.74 s on lock 0x7b540011a3d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:33.847001 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6297999616}
W20260709 13:05:33.907325 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.903774 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 56805.1us
W20260709 13:05:33.907918 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.820518 22118 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:33.912983 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45444.9us
I20260709 13:05:33.913899 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:33.915584 15810 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:33.916556 22044 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:33.917047 22044 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.917451 22044 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:33.888439 22287 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:34.004868 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.007839 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.901772 22222 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:33.907344 22183 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:33.924576 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:34.032200 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 311058us
I20260709 13:05:33.904764 21833 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:34.050406 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:33.920337 22041 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:34.054970 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.060204 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:33.924065 22189 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.032470 22029 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.841059 21930 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.209s
I20260709 13:05:34.031559 22027 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:34.077592 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 252078us
I20260709 13:05:34.078043 15845 raft_consensus.cc:1240] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:33.904639 22203 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.045496 22261 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.919605 22046 consensus_queue.cc:1048] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.143s
I20260709 13:05:34.072822 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:34.117264 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 233621us
I20260709 13:05:34.127441 21731 raft_consensus.cc:2764] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:34.127948 14979 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.128631 21833 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:34.114106 15159 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 12.
I20260709 13:05:34.071666 21942 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:34.075555 22260 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:05:34.075474 22253 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:34.257189 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.239960 15163 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 24.
W20260709 13:05:34.189448 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.256649 15082 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:05:34.296665 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.203837 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 296273us
W20260709 13:05:34.286101 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 225988us
W20260709 13:05:34.261188 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78076.7us
I20260709 13:05:34.303251 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.079783 22041 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:34.116765 22283 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:05:34.218533 22200 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
W20260709 13:05:34.321615 15128 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.025s (SENT)
I20260709 13:05:34.293082 22253 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.102599 22183 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:34.259471 22230 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:34.123869 22182 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:34.327783 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.263172 22227 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.222075 22198 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:34.246713 22203 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.294178 22250 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:34.265774 21895 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:34.324753 22037 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:34.094965 22029 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.118356 22261 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.123620 22189 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.334434 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 256043us
I20260709 13:05:34.328792 22253 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:34.334371 22118 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:34.257938 22263 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.329262 22183 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:34.346238 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 89202.3us
I20260709 13:05:34.217128 22085 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.264s
I20260709 13:05:34.327422 22200 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:34.317505 21942 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.341353 22046 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:34.343291 22037 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.361389 22308 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.362120 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.362004 22308 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.364391 22308 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.366695 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.367448 15845 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
W20260709 13:05:34.369156 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 46746.5us
I20260709 13:05:34.369928 21942 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.371444 22263 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:34.375046 22286 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.862s	user 0.004s	sys 0.008s
I20260709 13:05:34.377811 22101 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:34.378420 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.379022 15128 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.588s (SENT)
I20260709 13:05:34.379411 15128 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.352711 22183 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.382256 15157 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:34.385726 15232 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:05:34.380653 15128 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 2.104s (SENT)
I20260709 13:05:34.386027 15126 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.386253 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1703) took 1223 ms. Trace:
I20260709 13:05:34.384210 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:34.389365 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.399325 22261 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.401410 22033 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.403791 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:34.405411 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48286.5us
I20260709 13:05:34.406788 15054 leader_election.cc:400] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19.
I20260709 13:05:34.357609 22037 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:34.407629 22317 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:34.407724 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:34.405455 22264 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:05:34.409318 22264 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.408483 22317 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:34.408517 22036 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:05:34.386665 15842 rpcz_store.cc:276] 0709 13:05:33.162998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.571366 (+408368us) service_pool.cc:224] Handling call
0709 13:05:33.571655 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.571921 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:33.720971 (+149050us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.385844 (+664873us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b1800255040. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d3f9dc060 00007f0d3f9ee716 00007f0d3f9ff287
0709 13:05:34.385938 (+    94us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.385990 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:34.386037 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.386102 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.386203 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9563,"prepare.queue_time_us":307243,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":76813,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":23873,"spinlock_wait_cycles":36534528,"thread_start_us":693703,"threads_started":3,"wal-append.queue_time_us":562064}
I20260709 13:05:34.348580 22198 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.410226 22289 consensus_meta.cc:287] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:05:34.420522 22319 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:34.421411 22184 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:34.421801 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.422482 15086 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:34.422209 22253 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.425635 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.428791 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:34.432199 15159 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:34.431247 15160 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:05:34.430755 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 551751us
W20260709 13:05:34.436890 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 52722.6us
I20260709 13:05:34.437961 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.439105 15229 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.410737 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.480690 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:34.481464 15156 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:34.481837 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43731.6us
W20260709 13:05:34.480909 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 43803.4us
I20260709 13:05:34.482980 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:34.483153 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.418509 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.482975 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:34.484041 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.424582 22253 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:34.484932 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 126147us
I20260709 13:05:34.489794 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.490830 15201 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.419673 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3156) took 1338 ms. Trace:
I20260709 13:05:34.490852 15126 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:34.487741 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 415076us
I20260709 13:05:34.491104 21042 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:34.493625 21042 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:34.494109 21042 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:34.486779 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1545) took 1150 ms. Trace:
I20260709 13:05:34.504144 15842 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:34.508512 15202 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:34.510699 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.522150 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.510762 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.491761 15155 rpcz_store.cc:276] 0709 13:05:33.081319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.081469 (+   150us) service_pool.cc:224] Handling call
0709 13:05:33.125472 (+ 44003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.373161 (+247689us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:33.412977 (+ 39816us) log.cc:844] Serialized 14 byte log entry
0709 13:05:33.576669 (+163692us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:33.619467 (+ 42798us) log.cc:844] Serialized 26 byte log entry
0709 13:05:33.634253 (+ 14786us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:33.846463 (+212210us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b1800497b60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:05:34.071506 (+225043us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.096430 (+ 24924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.096511 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.418979 (+322468us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b5400151920. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f959dbe 00007f0d3f964930 00007f0d3f9648d8 00007f0d3f964858 00007f0d3f964813 00007f0d3f963a9c 00007f0d4584f626 00007f0d45849dd8 00007f0d3f9a07cb 00007f0d3f9a32c0
0709 13:05:34.419461 (+   482us) raft_consensus.cc:1685] finished
0709 13:05:34.419482 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.419616 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51653,"prepare.queue_time_us":35212,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":78731,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":22395,"spinlock_wait_cycles":21318144,"thread_start_us":300046,"threads_started":2,"wal-append.queue_time_us":123275}
I20260709 13:05:34.524040 15202 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.525274 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.482057 22261 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:34.511706 15844 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:34.403846 22033 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.488742 22230 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:34.548019 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 129578us
I20260709 13:05:34.418663 22101 raft_consensus.cc:2764] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:34.417678 22264 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.534049 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:34.497673 15235 rpcz_store.cc:276] 0709 13:05:33.336713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.348261 (+ 11548us) service_pool.cc:224] Handling call
0709 13:05:33.852139 (+503878us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:33.852321 (+   182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.852660 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:34.097758 (+245098us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.349917 (+252159us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.383217 (+ 33300us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.383407 (+   190us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:34.383443 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.383510 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.440983 (+ 57473us) raft_consensus.cc:1685] finished
0709 13:05:34.486582 (+ 45599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.486720 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":200600,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":7770,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":1908,"spinlock_wait_cycles":5251840,"thread_start_us":469781,"threads_started":2,"wal-append.queue_time_us":278930}
I20260709 13:05:34.430094 22029 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.489470 22150 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:34.420205 22289 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.543633 22085 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:34.484912 22036 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:34.542732 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.411698 22317 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:34.544025 15162 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:34.502096 22319 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:34.570605 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.571329 15848 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:34.504065 22308 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:34.551036 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.430402 22041 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.552278 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.419670 22198 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.555160 22189 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.556636 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:34.543215 22323 log.cc:927] Time spent T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:05:34.591413 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:34.559428 15237 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:34.592674 15201 leader_election.cc:400] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:05:34.593149 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.532232 22315 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:34.593864 22315 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.594271 22315 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.594512 22117 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:34.595690 22289 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.550161 22203 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:34.414690 22044 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.538513 21042 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.563601 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 182962us
I20260709 13:05:34.603833 15157 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:34.542359 22109 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:34.542204 21948 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:34.535846 21895 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:05:34.416812 22287 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.565847 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48497.6us
I20260709 13:05:34.570600 22317 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:34.567619 15847 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:34.568797 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:34.561501 22324 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:34.550405 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.550545 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.572263 15232 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:34.573925 15231 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:34.621026 15233 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:34.575814 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.621760 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.622246 15235 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 21: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:34.577621 22037 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.583308 22319 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:34.584954 22292 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:34.559720 22150 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:34.560665 22184 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:05:34.542265 22200 consensus_queue.cc:1059] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:05:34.591619 22085 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:34.593495 15162 pending_rounds.cc:85] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 11
I20260709 13:05:34.563241 22263 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.598695 15840 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:34.601939 15159 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:34.604166 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:34.553399 22149 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.029s	sys 0.040s
I20260709 13:05:34.606477 15229 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:34.608054 21042 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.624290 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.624658 15232 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:34.565672 22033 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:05:34.602810 22203 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.598764 22308 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:34.573290 22036 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.608196 22289 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.638746 15845 raft_consensus.cc:2368] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17.
I20260709 13:05:34.641857 22037 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.642431 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.642992 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.643121 22319 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.644470 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:34.646351 22263 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.649451 22253 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.658349 15233 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:34.662204 15156 raft_consensus.cc:2410] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:05:34.662983 15158 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:34.657512 15127 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:34.621959 22324 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.667316 15201 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.599076 21929 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:05:34.685860 15154 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:34.635871 15162 pending_rounds.cc:107] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:34.690680 15162 raft_consensus.cc:2889] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.634709 22150 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:34.608946 22198 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:05:34.701789 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:34.654299 22149 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.029s	sys 0.041s
I20260709 13:05:34.676678 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.677225 15234 raft_consensus.cc:2368] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17.
I20260709 13:05:34.678193 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.687602 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:34.670146 22203 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:34.634253 22189 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:05:34.694033 22101 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.694536 22200 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:34.694461 22253 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.695407 22263 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.696686 22287 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.697314 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.704550 15232 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:34.689957 22222 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:34.705404 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.707741 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:34.713709 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:34.668210 21942 consensus_queue.cc:1059] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:05:34.688536 22324 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:34.688336 22330 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.718329 15128 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.863s (SENT)
I20260709 13:05:34.729053 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.684325 22289 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.742367 22289 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.730158 15847 raft_consensus.cc:2368] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:05:34.744472 15848 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:34.716354 22200 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:34.756630 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 68950.8us
I20260709 13:05:34.714871 22317 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:34.766335 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 89740.8us
I20260709 13:05:34.728732 22322 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:34.747350 22044 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.766674 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:34.769447 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:34.773267 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.755781 22222 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:34.774310 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.756717 22292 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.782418 22292 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.782692 22222 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.758728 22200 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:34.785244 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.786034 15229 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:34.786955 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.782555 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.787668 15229 raft_consensus.cc:2393] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:34.789073 22139 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:34.789726 22139 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:34.790158 22139 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:34.792546 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.789749 22118 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:34.736069 22101 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.789255 15080 raft_consensus.cc:1240] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: [18.14-19.15]
I20260709 13:05:34.794265 22036 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:34.793231 15082 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:34.796986 22118 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.797506 22118 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.775023 22308 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.783201 22149 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:34.804915 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.778724 15233 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:34.793021 21942 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:34.807593 15201 leader_election.cc:400] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:05:34.808310 15201 leader_election.cc:403] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:05:34.808676 15232 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:34.788610 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:34.809664 15201 leader_election.cc:400] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17.
I20260709 13:05:34.810123 15201 leader_election.cc:403] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:05:34.833467 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.832921 15229 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:05:34.836583 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47487.5us
I20260709 13:05:34.837560 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.848634 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:34.849373 15233 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 13.
W20260709 13:05:34.849687 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44894.2us
I20260709 13:05:34.850478 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.853627 15081 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:34.856892 15084 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:34.854689 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.812731 22149 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:34.746533 22330 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:34.883098 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45660.3us
I20260709 13:05:34.865751 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:34.866459 22333 log.cc:927] Time spent T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:05:34.866348 15088 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:34.870121 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.884369 15126 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.870651 22036 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:34.873521 22101 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.873728 22118 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:34.874642 22131 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:34.876034 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.834030 22317 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:34.878346 22336 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:34.838217 22325 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:34.880146 15083 raft_consensus.cc:1240] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:05:34.880089 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:34.888931 15079 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:34.831214 21948 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 14. Current term is 15.
I20260709 13:05:34.729115 22085 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:34.927652 15085 raft_consensus.cc:2393] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:34.924701 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:34.941569 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.919149 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.930182 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.942483 15084 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:34.942605 15088 raft_consensus.cc:3060] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:34.931370 15163 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 3158) took 1277 ms (client timeout 1644 ms). Trace:
W20260709 13:05:34.945255 15163 rpcz_store.cc:269] 0709 13:05:33.645841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.672160 (+ 26319us) service_pool.cc:224] Handling call
0709 13:05:34.922850 (+1250690us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400086bd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:34.923015 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437631744}
I20260709 13:05:34.882764 22149 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:34.938381 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58380.8us
I20260709 13:05:34.857781 22046 raft_consensus.cc:2764] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:34.952029 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:34.887456 22061 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:34.928645 22036 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:34.929973 22131 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.968685 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.882465 22182 consensus_queue.cc:1048] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:05:34.975800 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.979111 15083 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:34.933871 22101 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:34.937621 22336 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.921173 21942 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:05:34.940811 22320 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:34.922171 22027 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
W20260709 13:05:34.987426 15113 heartbeater.cc:646] Failed to heartbeat to 127.13.163.254:45133 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.13.163.254:45133 timed out after 15.000s (SENT)
I20260709 13:05:34.963925 22061 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:34.987910 22331 log.cc:927] Time spent T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:05:34.999539 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 292636us
I20260709 13:05:34.993805 22249 raft_consensus.cc:3055] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:34.984896 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:34.995198 22315 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:34.974503 22036 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:34.977270 22131 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:35.017333 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:35.016429 15082 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:35.011511 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.012418 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:34.980142 22222 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:35.014115 15079 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:34.985882 22044 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:34.985504 22336 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:35.021538 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.023581 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.023963 15084 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:35.026348 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:35.026365 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 101763us
I20260709 13:05:35.028326 15085 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:35.019513 15087 raft_consensus.cc:2376] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17. Re-sending same reply.
I20260709 13:05:35.035894 15839 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:35.021625 22183 consensus_meta.cc:287] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.004s
I20260709 13:05:35.040583 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.003253 22249 raft_consensus.cc:740] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:34.997766 22101 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.044201 22200 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.044574 22183 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.045091 22044 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:35.109921 22044 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.049664 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:35.049896 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.051437 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.070847 15088 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
W20260709 13:05:35.080319 22261 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.004s
I20260709 13:05:35.076689 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:35.083105 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.082248 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:35.102396 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:35.141556 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 189635us
I20260709 13:05:35.144287 15082 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:35.112115 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.144616 15087 raft_consensus.cc:3060] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:35.145581 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.147948 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:35.148005 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 65850.3us
I20260709 13:05:35.140753 15083 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:05:35.147909 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 194126us
I20260709 13:05:35.154038 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.081203 22250 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.033144 22139 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.103281 22222 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:35.167639 15079 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:35.152308 22261 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.175601 22222 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.168936 22341 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:35.177430 22341 raft_consensus.cc:697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:35.169286 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:35.169503 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:35.156016 22112 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:35.174532 22139 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.144692 22184 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.179406 15230 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 14.
W20260709 13:05:35.180099 14972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 210386us
W20260709 13:05:35.199891 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54378.8us
I20260709 13:05:35.114529 22182 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:35.200868 15051 leader_election.cc:400] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17.
I20260709 13:05:35.201189 15051 leader_election.cc:403] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:05:35.178540 22101 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.217784 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:35.218457 15156 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.216182 15810 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:35.232199 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78192.3us
I20260709 13:05:35.178387 22341 consensus_queue.cc:237] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.242408 15813 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.251674 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.215982 21895 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:35.230702 22101 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.259438 15083 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:35.259967 15082 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:35.260324 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 208988us
W20260709 13:05:35.260490 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44404us
I20260709 13:05:35.267498 15156 raft_consensus.cc:2468] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
W20260709 13:05:35.267994 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47284.6us
I20260709 13:05:35.207779 22184 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.279524 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.288316 15082 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:35.251792 22250 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17.
I20260709 13:05:35.257472 21895 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:35.304642 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.259192 22308 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:35.308746 15814 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:35.287874 22346 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:35.316244 15083 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
W20260709 13:05:35.319262 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 85886.8us
I20260709 13:05:35.324254 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.328339 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:35.263180 22061 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.306932 22335 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:35.312278 22101 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.312494 21895 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:35.260022 22352 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:05:35.348053 22373 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:35.369416 22373 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.369670 22320 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.356220 22253 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.370630 22354 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.371186 22253 raft_consensus.cc:2804] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:35.330004 22342 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.371919 22253 raft_consensus.cc:697] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:35.350009 22183 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.372336 22342 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.373308 22183 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.373852 22183 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.374748 22342 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.336575 22308 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.358152 15083 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:35.320567 22346 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:35.376534 22346 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:35.341475 22335 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:35.336443 22203 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.616s	user 0.005s	sys 0.005s
I20260709 13:05:35.361989 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.368499 22200 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:35.369931 22344 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:05:35.371249 22354 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.286432 22131 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:35.330641 22324 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
W20260709 13:05:35.364827 22349 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:05:35.411016 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.415195 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.572937 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.573623 15088 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:35.377141 22308 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.428925 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:35.454495 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 253771us
I20260709 13:05:35.577373 15846 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:35.551841 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 227703us
W20260709 13:05:35.523887 22325 consensus_meta.cc:287] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.003s
I20260709 13:05:35.407981 22336 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:35.476866 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 48298.3us
I20260709 13:05:35.369948 22373 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:35.377797 22101 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.578625 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:35.579170 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 168218us
I20260709 13:05:35.579530 15082 raft_consensus.cc:2468] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:35.578884 21929 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:35.579865 15813 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.580209 21929 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:35.414575 22203 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.580708 21929 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:35.580459 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.582412 22200 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.454720 22324 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.584331 15234 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:35.477735 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:35.577718 15051 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.585362 22324 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.586014 22139 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:35.586206 15051 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.457880 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.581338 15228 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:35.588364 22139 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:35.588699 15083 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:35.591825 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.598385 15088 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:35.492503 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:35.593087 22325 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17.
I20260709 13:05:35.456962 22253 consensus_queue.cc:237] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.611474 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.626742 15202 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.632298 15087 raft_consensus.cc:2468] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:35.543437 22184 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:35.652668 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 72863.7us
W20260709 13:05:35.617254 22149 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.003s
I20260709 13:05:35.606472 22308 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.585224 22203 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.654392 22085 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:35.655797 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.711375 15088 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:35.669574 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:35.624109 22139 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.665145 22061 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.674489 22367 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.643608 22183 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.676085 22046 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:35.677028 22375 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:35.655463 22117 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.677587 22370 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:35.655722 22044 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.673491 22184 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.678663 22336 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.678285 22383 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:05:35.636417 22324 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:05:35.676668 22260 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.680292 15237 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:35.666616 21895 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.668488 22378 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:35.677498 22149 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.683846 22364 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.217s
W20260709 13:05:35.685774 22368 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:05:35.687264 22036 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.659s	user 0.006s	sys 0.005s
W20260709 13:05:35.685102 22372 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:05:35.739337 22046 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.741374 21042 raft_consensus.cc:3055] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:05:35.743360 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73881.6us
I20260709 13:05:35.775022 21042 raft_consensus.cc:740] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:35.636087 22319 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:05:35.610255 22351 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.359s
I20260709 13:05:35.654877 22362 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.330s
I20260709 13:05:35.779474 21042 consensus_queue.cc:260] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.721617 22085 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:35.781445 22085 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.782071 22149 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.767414 22378 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:35.783885 22378 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.738418 21833 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:35.756229 22150 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 1.058s	user 0.012s	sys 0.002s
I20260709 13:05:35.785660 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:35.785514 22150 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.787928 22150 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.754922 22131 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.788331 22046 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:35.720594 22139 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.789305 15231 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 20: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:35.791005 22139 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.775418 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:35.794435 21895 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:35.794862 22249 consensus_queue.cc:260] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.795466 15128 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:35.767756 22260 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.796043 22249 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:35.797253 22383 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:35.797698 22383 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.798020 22260 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.755146 22044 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.798094 22383 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:35.800912 22044 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:35.780578 21042 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:35.780603 22362 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:05:35.802894 22290 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:35.803333 22290 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:35.803848 15236 raft_consensus.cc:2468] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:35.773988 22036 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.804705 22101 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.806753 22036 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.753635 22375 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.804234 22290 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:35.808969 22375 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.753806 22370 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.810920 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.811324 22370 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.811570 15085 raft_consensus.cc:2376] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17. Re-sending same reply.
I20260709 13:05:35.818384 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1720) took 1317 ms. Trace:
I20260709 13:05:35.818879 15839 rpcz_store.cc:276] 0709 13:05:34.501148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.631890 (+130742us) service_pool.cc:224] Handling call
0709 13:05:34.664141 (+ 32251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.037751 (+373610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.818140 (+780389us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54005fd554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:35.818317 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854340480}
I20260709 13:05:35.824323 15088 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:35.812463 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.825843 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.841302 15088 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:35.849969 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.853282 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:35.858611 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 123244us
I20260709 13:05:35.860438 15088 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:35.833168 22379 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.812337 22346 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:35.883893 15127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.012s	sys 0.019s
I20260709 13:05:35.864678 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:35.884325 15127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.012s	sys 0.019s
I20260709 13:05:35.884625 15085 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:35.885057 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.11839e+06us
I20260709 13:05:35.886014 15127 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.893602 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:35.895188 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:35.856590 22389 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.901826 22389 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.881064 22222 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:35.904130 22389 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.904443 22222 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:35.863741 21929 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.905148 22379 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.903714 22373 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:35.910997 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:35.911940 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:35.918419 22292 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.920639 21929 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.921581 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:35.908442 22319 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.908792 22222 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:35.926374 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 219206us
I20260709 13:05:35.927637 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1666) took 1500 ms. Trace:
I20260709 13:05:35.930022 15841 raft_consensus.cc:2393] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:35.931599 15159 raft_consensus.cc:1240] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:35.929833 15157 rpcz_store.cc:276] 0709 13:05:34.427509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.481986 (+ 54477us) service_pool.cc:224] Handling call
0709 13:05:34.510134 (+ 28148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.605540 (+ 95406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.927393 (+1321853us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b540017e154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:35.927578 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215084928}
W20260709 13:05:35.932955 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.935825 15083 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:35.936926 15088 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:35.937883 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:35.938489 15088 raft_consensus.cc:2376] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18. Re-sending same reply.
I20260709 13:05:35.963086 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.966348 15083 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:35.973027 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.976805 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:35.980923 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:35.974304 22189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.014s	sys 0.050s
I20260709 13:05:35.982496 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:35.983750 15054 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:35.939675 22260 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:35.993635 15157 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:35.980386 22292 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.988044 22373 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:35.988566 22379 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.990588 22330 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:35.994904 15085 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:35.994601 22390 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:36.006479 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1664) took 1956 ms. Trace:
I20260709 13:05:36.006933 15158 rpcz_store.cc:276] 0709 13:05:34.049919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.050079 (+   160us) service_pool.cc:224] Handling call
0709 13:05:36.006265 (+1956186us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540030cd54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.006410 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386450688}
I20260709 13:05:36.016902 15083 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:36.018100 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:36.018438 22385 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.278s	user 0.003s	sys 0.000s
I20260709 13:05:36.021121 22325 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:36.021683 22325 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:36.022697 22325 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.002521 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.004149 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.025625 22290 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:35.997243 22260 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.030081 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:36.030678 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47008.5us
I20260709 13:05:36.031348 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1561) took 1145 ms. Trace:
W20260709 13:05:35.995652 22189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.015s	sys 0.050s
I20260709 13:05:36.027738 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.036405 15088 raft_consensus.cc:2410] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:05:36.031700 15230 rpcz_store.cc:276] 0709 13:05:34.885743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.357349 (+471606us) service_pool.cc:224] Handling call
0709 13:05:35.378239 (+ 20890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.378602 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.378636 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.408365 (+ 29729us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.531601 (+123236us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:35.531648 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.531729 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.017688 (+485959us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:36.027129 (+  9441us) raft_consensus.cc:1685] finished
0709 13:05:36.027152 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.031286 (+  4134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2394,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":11138,"spinlock_wait_cycles":35790720,"thread_start_us":39526,"threads_started":1,"wal-append.queue_time_us":267626}
I20260709 13:05:36.035200 22325 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:36.029002 22390 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:36.040359 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.038317 22373 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:36.041851 15085 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:05:36.042246 22373 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.043026 15085 raft_consensus.cc:1240] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:36.046394 22361 consensus_queue.cc:1059] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:36.048219 22373 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:36.050941 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:36.051820 15086 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:36.057590 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.059015 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.060427 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.061261 15845 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 14.
I20260709 13:05:36.062832 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.065343 15846 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:36.068933 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.075101 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.038655 22292 consensus_queue.cc:1059] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:36.080338 22292 consensus_queue.cc:1243] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:36.080858 22292 raft_consensus.cc:3055] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:36.079595 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.082880 15230 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:36.084599 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.072963 22189 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.085877 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.086440 15235 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 14.
I20260709 13:05:36.086526 15842 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:36.086719 22289 raft_consensus.cc:3055] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:36.087198 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.085712 22189 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.088274 15160 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:36.088945 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3168) took 1568 ms. Trace:
I20260709 13:05:36.089217 22189 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.089272 15160 rpcz_store.cc:276] 0709 13:05:34.520052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.800241 (+280189us) service_pool.cc:224] Handling call
0709 13:05:36.087889 (+1287648us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b580013f298. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:36.088081 (+   192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.088685 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.088891 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9696128}
I20260709 13:05:36.094334 15845 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:36.096450 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1559) took 1211 ms. Trace:
I20260709 13:05:36.096932 15231 rpcz_store.cc:276] 0709 13:05:34.885129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.049384 (+164255us) service_pool.cc:224] Handling call
0709 13:05:36.096217 (+1046833us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54002d3f54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.096360 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474850176}
I20260709 13:05:36.098238 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.099352 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.099915 15231 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:36.101869 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2737) took 1436 ms. Trace:
I20260709 13:05:36.102425 15845 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:36.105345 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.105989 15839 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:36.107036 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.107383 22383 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.108279 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2552) took 1247 ms. Trace:
I20260709 13:05:36.108645 15079 rpcz_store.cc:276] 0709 13:05:34.860393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.023242 (+162849us) service_pool.cc:224] Handling call
0709 13:05:36.108123 (+1084881us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54003cf354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.108225 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271518976}
I20260709 13:05:36.109354 15813 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:36.109684 22383 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.096725 15842 raft_consensus.cc:1240] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:36.111176 15813 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:36.112782 22346 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.113634 22353 raft_consensus.cc:2804] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:36.119378 22353 raft_consensus.cc:697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.119633 15086 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:36.120385 22353 consensus_queue.cc:237] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.122123 15082 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:05:36.115314 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1555) took 1560 ms. Trace:
I20260709 13:05:36.124190 15082 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.124326 15228 rpcz_store.cc:276] 0709 13:05:34.554408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.678130 (+123722us) service_pool.cc:224] Handling call
0709 13:05:34.678409 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.588329 (+909920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.115050 (+526721us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400295254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:36.115245 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076087552}
I20260709 13:05:36.116621 22182 raft_consensus.cc:2804] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:36.126784 22182 raft_consensus.cc:697] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.127767 22182 consensus_queue.cc:237] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.131819 22222 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.135823 15234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2730) took 1546 ms. Trace:
I20260709 13:05:36.101912 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.139292 15848 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:36.140184 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:36.102262 15232 rpcz_store.cc:276] 0709 13:05:34.665637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.732690 (+ 67053us) service_pool.cc:224] Handling call
0709 13:05:34.808477 (+ 75787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.813000 (+  4523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.101678 (+1288678us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b540005bc50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.101798 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173457280}
I20260709 13:05:36.142046 15231 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:36.136327 15234 rpcz_store.cc:276] 0709 13:05:34.589225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.732680 (+143455us) service_pool.cc:224] Handling call
0709 13:05:34.733157 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.584775 (+851618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.135506 (+550731us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54003d9ad4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:36.135745 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957184512}
I20260709 13:05:36.081167 22292 raft_consensus.cc:740] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:36.149938 15840 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 13:05:36.150104 15237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2733) took 1559 ms (client timeout 1503 ms). Trace:
I20260709 13:05:36.150386 15233 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:05:36.150604 15237 rpcz_store.cc:269] 0709 13:05:34.590324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.592913 (+  2589us) service_pool.cc:224] Handling call
0709 13:05:36.149900 (+1556987us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b5400294d54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.150038 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347084544}
I20260709 13:05:36.160133 15848 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:36.161087 22292 consensus_queue.cc:260] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.162179 22292 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:36.162194 22335 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.163622 22335 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:36.165129 22260 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.107138 22315 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 13 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:36.148221 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.171598 22352 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:36.172169 22352 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.176591 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:36.087212 22289 raft_consensus.cc:740] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.182660 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:36.185578 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.189210 22289 consensus_queue.cc:260] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.190053 21929 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:36.190627 21929 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.191072 22289 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:36.170292 22263 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:36.193289 22263 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.194219 21929 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:36.187176 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.187901 22085 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.173400 22260 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.195322 22263 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.170526 22264 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.195576 22085 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.195668 22260 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.196276 22264 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.197710 22085 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.198132 22260 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:36.198467 22264 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:36.186331 22325 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.203136 14979 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 16, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:36.194155 21942 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:36.210182 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.213408 15080 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:36.210640 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:36.206652 15813 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:36.214380 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.215197 15080 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:36.218946 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.219715 15229 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:36.221333 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:36.222158 15080 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:36.224727 15080 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:36.201522 22315 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:36.234406 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.235615 15234 raft_consensus.cc:2410] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:05:36.237650 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.239704 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.240628 15231 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:36.250095 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.250456 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.250794 15843 raft_consensus.cc:3060] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:36.211352 21942 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.253063 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.253777 15230 raft_consensus.cc:2393] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:36.254040 15229 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:36.250129 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.218724 22322 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:36.252346 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.173149 22352 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.257580 15841 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:36.259482 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.261411 15228 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:36.260669 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.262758 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.263657 15237 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 13.
I20260709 13:05:36.261780 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.201550 22325 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.265985 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.230185 22222 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.272255 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:36.273177 15162 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.269630 15158 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:36.303893 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:36.274451 15160 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:36.306087 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.307492 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:36.274369 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.275741 15156 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.277400 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.278409 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.283643 22029 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.285405 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.313028 15847 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:36.313412 22029 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.290180 22227 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:36.314023 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:36.314510 22227 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.315147 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.316079 22029 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:36.316218 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.316895 15232 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:36.259232 22377 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.508s	user 0.003s	sys 0.001s
I20260709 13:05:36.288540 22361 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:36.297197 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:36.317875 15161 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:36.317720 22361 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.318563 15157 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:36.320008 22361 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.296856 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:36.299852 15088 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:36.301869 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:36.320828 15155 raft_consensus.cc:2368] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:36.313786 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.321107 15230 raft_consensus.cc:3060] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:36.291640 22346 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.321698 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.321923 22346 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.322731 15154 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:36.324039 22346 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.297703 15228 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:36.325654 15228 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:36.325766 15088 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:05:36.327024 15088 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.327630 15228 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:36.328184 15088 raft_consensus.cc:1275] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:36.312443 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.329255 15156 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:36.316687 22227 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:36.330540 15232 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:36.332180 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.332368 15232 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:36.321013 15158 raft_consensus.cc:2376] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 24. Re-sending same reply.
I20260709 13:05:36.334031 15088 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:36.338356 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.338981 15840 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:36.237418 22373 raft_consensus.cc:2804] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:36.333112 15230 raft_consensus.cc:2468] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
W20260709 13:05:36.348415 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.348824 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 74547.3us
I20260709 13:05:36.349970 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:36.222527 22264 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:05:36.255326 22315 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.262938 22322 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:36.345055 15051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.503s (SENT) [suppressed 21 similar messages]
I20260709 13:05:36.353143 15842 pending_rounds.cc:85] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 12
I20260709 13:05:36.354190 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.364959 15842 pending_rounds.cc:107] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:36.365912 15842 raft_consensus.cc:2889] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:36.364903 15228 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:36.364964 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.249872 22378 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.102s
I20260709 13:05:36.368139 15162 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:36.371016 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.355865 15157 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:36.351826 21942 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:36.361922 22112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.146s	user 0.027s	sys 0.035s
I20260709 13:05:36.361821 15843 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:36.361327 15840 raft_consensus.cc:2410] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:36.352983 22361 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:36.375823 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:36.363534 22029 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:36.387199 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.387630 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.358466 22362 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.364141 22373 raft_consensus.cc:697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.366645 22322 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:36.418664 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64569.3us
I20260709 13:05:36.410264 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3234) took 1131 ms. Trace:
I20260709 13:05:36.419641 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.403518 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.409644 15840 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:36.370548 22222 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:36.411160 15083 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:05:36.422096 15845 raft_consensus.cc:2368] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:36.419649 15087 rpcz_store.cc:276] 0709 13:05:35.279505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.633380 (+353875us) service_pool.cc:224] Handling call
0709 13:05:35.633871 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:35.792244 (+158373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.792284 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:35.792988 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 13:05:35.926650 (+133662us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:35.926691 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.396010 (+469319us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400349754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:36.396078 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.396151 (+    73us) raft_consensus.cc:1685] finished
0709 13:05:36.396169 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.410193 (+ 14024us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10423,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":363163904,"thread_start_us":133252,"threads_started":1,"wal-append.queue_time_us":204277}
I20260709 13:05:36.423465 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:36.426472 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:36.428799 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:36.429523 15161 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:36.394591 21942 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:36.432996 22315 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.426934 22361 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:05:36.421957 22222 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.436769 22222 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:36.369308 22253 consensus_queue.cc:1048] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:05:36.373893 22353 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:05:36.440953 22383 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:36.441497 22383 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.441970 22383 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:36.444777 22373 consensus_queue.cc:237] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:36.382814 22112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.027s	sys 0.035s
I20260709 13:05:36.447086 22112 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.383028 22346 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:05:36.448236 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.384068 22263 consensus_queue.cc:1059] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:05:36.383472 22189 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:36.449218 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:36.449466 22059 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.450042 22059 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.450505 22189 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:36.451136 22189 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.451165 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.452033 15843 raft_consensus.cc:2393] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:36.452349 22059 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.452903 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.379495 22227 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:36.454630 15235 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:36.405288 22264 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.452116 22189 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.459730 15844 raft_consensus.cc:2393] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:36.448097 22112 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.387306 22330 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:05:36.475198 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:36.476903 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.478045 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2740) took 1718 ms. Trace:
I20260709 13:05:36.450451 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.478446 15233 rpcz_store.cc:276] 0709 13:05:34.759284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.875209 (+115925us) service_pool.cc:224] Handling call
0709 13:05:36.150037 (+1274828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.150818 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.477827 (+327009us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400294d50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.477976 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930904448}
I20260709 13:05:36.441555 22353 raft_consensus.cc:3055] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:36.459210 15841 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:36.486269 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.481173 22370 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:36.483322 15810 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:36.494980 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.496950 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.467109 22322 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.498466 21929 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:36.533521 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 107128us
I20260709 13:05:36.427691 22287 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:05:36.535212 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:36.479321 22352 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:36.534241 15840 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:36.487650 22353 raft_consensus.cc:740] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.481523 22037 consensus_queue.cc:1059] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:05:36.422889 22029 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.431105 22362 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.563814 22322 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.546610 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:36.529338 22330 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:36.617122 22330 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.618475 22330 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.620232 22330 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:36.535355 21931 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:36.542342 22366 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:36.624555 21931 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.625303 21931 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:36.544291 22289 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:36.534987 22027 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:36.478883 22264 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.541754 21929 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.614488 22037 consensus_queue.cc:1243] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:36.649837 22222 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.651376 15085 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:36.651785 22362 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.652249 22222 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.569413 21942 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:05:36.678203 21942 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.681411 22029 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.681243 21942 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.682024 22029 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:36.682581 22029 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.683616 21942 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.684243 21942 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:36.694507 15086 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:05:36.709399 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41710 (request call id 1562) took 1823 ms (client timeout 1590 ms). Trace:
I20260709 13:05:36.723727 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:36.746340 15236 rpcz_store.cc:269] 0709 13:05:34.886048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.016985 (+130937us) service_pool.cc:224] Handling call
0709 13:05:36.709205 (+1692220us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400058cd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:36.709330 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477365632}
I20260709 13:05:36.706699 15083 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.677429 22289 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:36.751642 15087 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:36.684782 22029 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:36.756429 22412 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 13:05:36.758107 22411 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.105s	user 0.001s	sys 0.001s
I20260709 13:05:36.760349 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1693) took 1102 ms. Trace:
I20260709 13:05:36.760805 15079 rpcz_store.cc:276] 0709 13:05:35.658110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.109582 (+451472us) service_pool.cc:224] Handling call
0709 13:05:36.109899 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.110277 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.110306 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.110795 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.310227 (+199432us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:36.310271 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.310347 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.757490 (+447143us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:36.760069 (+  2579us) raft_consensus.cc:1685] finished
0709 13:05:36.760090 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.760290 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36012,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":18262,"spinlock_wait_cycles":12171136,"thread_start_us":199038,"threads_started":1,"wal-append.queue_time_us":527811}
I20260709 13:05:36.709679 14979 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 13, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 13 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.767824 22383 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.769963 21929 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:36.771158 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.747332 15082 raft_consensus.cc:2410] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:05:36.685669 22354 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.685040 21942 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.706861 22222 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:36.675072 22027 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 13:05:36.775496 22289 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.760144 21931 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.770428 22353 consensus_queue.cc:260] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.796342 22222 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:36.824594 22383 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:36.829031 22394 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:36.849052 21942 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.851137 21942 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:36.860962 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:36.867475 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 413090us
I20260709 13:05:36.857353 22289 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.868620 22037 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:36.871531 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 526457us
I20260709 13:05:36.866304 22352 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.876356 22354 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.894351 22408 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:36.894884 22408 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.895794 22408 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.897516 22150 raft_consensus.cc:3055] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:36.897987 22150 raft_consensus.cc:740] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:36.870358 22253 consensus_queue.cc:1059] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:05:36.874480 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:36.905402 22408 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:36.912055 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:36.914321 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 367767us
I20260709 13:05:36.886584 21931 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:36.878098 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 381276us
I20260709 13:05:36.927577 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:36.918236 15842 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:36.929291 22189 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:36.930084 22189 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:36.883855 22352 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.962250 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.914081 22361 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:36.899008 22150 consensus_queue.cc:260] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.960178 21929 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.880471 22222 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.901083 22354 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:36.960399 22389 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:36.919623 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:36.976239 22389 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:36.983531 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:36.983322 22389 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:36.984616 15841 raft_consensus.cc:2393] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:36.985160 22389 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:36.985704 22425 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:36.987257 22381 raft_consensus.cc:2764] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:05:37.000310 22112 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:05:37.002558 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.003911 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.007232 22264 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:36.980523 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53181.9us
W20260709 13:05:36.982362 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 62835.5us
I20260709 13:05:37.014654 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.016645 15163 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:36.978856 22150 raft_consensus.cc:3060] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.019729 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:37.021216 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.021431 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:37.022091 15163 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:37.021970 15162 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
W20260709 13:05:37.023102 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.026526 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.026372 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.023995 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:37.011287 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:37.028924 15162 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:37.029717 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.030637 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.032377 15162 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:05:37.032722 15162 pending_rounds.cc:85] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 12
I20260709 13:05:37.032950 15162 pending_rounds.cc:107] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:05:37.030467 15842 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:37.033208 15162 raft_consensus.cc:2889] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:36.977396 21929 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.036494 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.037699 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:37.013154 22112 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:37.039930 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:37.041370 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:37.050639 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.013510 22362 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:37.041844 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:37.013995 22352 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:37.075680 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:37.046728 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 350547us
I20260709 13:05:37.076771 15234 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.050983 15236 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:05:37.086951 15202 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.087728 15163 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:37.089715 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.090804 15840 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:37.093308 15840 raft_consensus.cc:1240] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:05:37.094703 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49740us
I20260709 13:05:37.095911 15235 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:36.981889 22429 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:05:37.079758 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.098424 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:37.098760 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:37.098940 15156 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:05:37.014631 22118 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:05:37.035398 22373 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:37.132567 15231 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:37.131552 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:37.131896 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.020753 22322 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:05:37.163107 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:37.139853 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54744.6us
I20260709 13:05:37.167922 15159 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
W20260709 13:05:37.181517 22416 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.531s	user 0.004s	sys 0.000s
I20260709 13:05:37.094054 22290 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:37.158268 15236 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:37.172159 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.172737 15160 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:37.186483 15162 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:37.173666 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2561) took 1310 ms. Trace:
W20260709 13:05:37.135072 15839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.021s	sys 0.081s
W20260709 13:05:37.187245 15839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.036s	sys 0.092s
I20260709 13:05:37.178681 22118 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:37.187978 22118 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:37.186842 15086 rpcz_store.cc:276] 0709 13:05:35.862752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.170806 (+308054us) service_pool.cc:224] Handling call
0709 13:05:36.694168 (+523362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.745515 (+ 51347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.173420 (+427905us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400191750. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:37.173586 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170493824}
I20260709 13:05:37.178735 22322 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:37.189509 22322 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:37.190080 22319 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:37.189321 22118 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.185611 22290 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:37.138159 15842 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:37.187060 22325 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:37.190925 22290 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:37.190485 22322 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:37.197779 15163 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:37.184356 15234 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:37.171725 15154 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:37.102914 15202 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.205212 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:37.254877 22420 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:05:37.254537 22112 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:37.255431 22426 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.292s	user 0.000s	sys 0.000s
I20260709 13:05:37.099365 22029 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.162s
I20260709 13:05:37.218526 22373 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:37.258460 22354 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:37.260224 21929 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:37.132460 22390 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.078315 22362 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:37.075659 22366 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:37.094465 22430 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:37.264981 22435 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.266714 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:37.334149 15848 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:37.267668 22415 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.614s	user 0.000s	sys 0.001s
I20260709 13:05:37.273061 15201 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:37.335460 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62480.9us
I20260709 13:05:37.275226 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2740) took 1010 ms. Trace:
I20260709 13:05:37.247400 15235 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:05:37.138631 22249 raft_consensus.cc:2804] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:37.279408 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 176577us
I20260709 13:05:37.190502 22319 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:37.101365 15203 leader_election.cc:400] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:37.365447 15203 leader_election.cc:403] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:37.195816 22118 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:37.163735 22253 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 13:05:37.292680 15840 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
W20260709 13:05:37.316692 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 292766us
W20260709 13:05:37.377516 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 276356us
I20260709 13:05:37.378646 15203 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.341001 22435 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:37.380129 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.381075 15159 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 24.
I20260709 13:05:37.331996 22390 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.324571 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.326306 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.281635 22263 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:37.336397 15157 rpcz_store.cc:276] 0709 13:05:36.264743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.272000 (+  7257us) service_pool.cc:224] Handling call
0709 13:05:37.275010 (+1003010us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b540006f254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:37.275149 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36135680}
I20260709 13:05:37.342717 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.344645 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:37.349530 15160 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:37.329432 22362 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:37.329571 22366 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.324214 22373 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:37.357272 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:37.333555 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 307665us
I20260709 13:05:37.499279 15154 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:37.501843 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.366060 15201 leader_election.cc:400] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:37.372473 15848 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:37.345916 21929 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:37.375186 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:37.378554 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:37.253341 22399 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.296792 15237 raft_consensus.cc:1275] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:37.419595 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:37.405169 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.419565 22029 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:37.470752 14979 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:37.471097 22438 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:05:37.445858 22249 raft_consensus.cc:697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:37.446511 22264 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 13:05:37.423966 21942 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
W20260709 13:05:37.494392 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 151765us
I20260709 13:05:37.464509 22433 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:37.485982 22085 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:37.511994 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 133419us
I20260709 13:05:37.565133 22317 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:37.498853 22366 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:37.496096 22443 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:37.473137 22260 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:05:37.586120 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.581085 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 205960us
I20260709 13:05:37.592600 22431 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:37.593269 22378 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:37.579690 22085 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:37.571986 22263 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:37.569696 22029 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.614499 22390 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:37.567821 22444 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
W20260709 13:05:37.621774 22446 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:05:37.624051 22435 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:37.565761 22354 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:05:37.493363 22319 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.631256 22249 consensus_queue.cc:237] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.637816 14979 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 16, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:37.639421 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 260820us
W20260709 13:05:37.659067 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 280655us
I20260709 13:05:37.664500 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:37.680490 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 178757us
I20260709 13:05:37.683523 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:37.666823 22413 log.cc:927] Time spent T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.014s	user 0.002s	sys 0.002s
I20260709 13:05:37.654870 22085 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.687533 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.687453 15203 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:37.580327 22399 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.684760 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:37.694409 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:37.678433 22390 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.675875 22227 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:37.677829 22443 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:37.649379 22431 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:37.686419 22440 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:05:37.651139 22378 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:37.695565 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:37.678278 22354 consensus_queue.cc:1243] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:37.689836 22433 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:37.687156 22253 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:37.703066 15202 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:37.690968 22319 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.719777 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.713835 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:37.727757 15087 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:37.708252 22317 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:37.731066 15082 raft_consensus.cc:1275] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:05:37.731506 15082 pending_rounds.cc:85] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 11
I20260709 13:05:37.731747 15082 pending_rounds.cc:107] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:05:37.731986 15082 raft_consensus.cc:2889] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.732309 15082 pending_rounds.cc:107] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:37.732542 15082 raft_consensus.cc:2889] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.689927 22389 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:05:37.737200 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.738142 15084 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:37.697706 21929 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:37.698295 22227 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:37.696496 22264 raft_consensus.cc:2804] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:37.741385 15088 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:37.729254 22433 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:37.766005 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.750075 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:37.766856 15086 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:37.739464 15154 raft_consensus.cc:2468] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:37.751333 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.768392 15083 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 24.
I20260709 13:05:37.769464 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.770331 22264 raft_consensus.cc:697] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:37.752283 22452 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:37.753468 22317 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.755201 15084 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:37.738615 22455 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:37.755869 22436 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.557s	user 0.000s	sys 0.021s
I20260709 13:05:37.692945 22182 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:05:37.760538 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.761564 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.783685 15085 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:37.705874 22378 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:37.766304 15079 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:37.749506 21929 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.709501 22037 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:05:37.781643 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:37.765547 22433 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:37.787267 15084 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:37.747673 22456 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:37.789888 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:37.783459 15082 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:37.791230 15085 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:37.791446 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2564) took 1921 ms. Trace:
I20260709 13:05:37.729557 22263 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.789014 21929 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.709852 22443 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.795536 15080 rpcz_store.cc:276] 0709 13:05:35.869506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.225667 (+356161us) service_pool.cc:224] Handling call
0709 13:05:36.225981 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.226686 (+   705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.226720 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.236926 (+ 10206us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.331102 (+ 94176us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:36.336055 (+  4953us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.336295 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.336363 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.471259 (+1134896us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:37.790376 (+319117us) raft_consensus.cc:1685] finished
0709 13:05:37.791088 (+   712us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.791327 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":342816,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":523,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7719,"spinlock_wait_cycles":8342272,"thread_start_us":55164,"threads_started":2,"wal-append.queue_time_us":384864}
I20260709 13:05:37.781386 22455 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:37.805063 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78404.1us
I20260709 13:05:37.805101 22444 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.772934 22431 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.806169 22444 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.807116 22431 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:37.804188 15087 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:37.813014 15086 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
W20260709 13:05:37.814853 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.793732 22435 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.799480 22354 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:37.782514 22289 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:05:37.793648 22456 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:37.799552 22222 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:37.824815 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1707) took 1379 ms. Trace:
I20260709 13:05:37.812093 22444 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:37.827939 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:37.825234 15081 rpcz_store.cc:276] 0709 13:05:36.445360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.459842 (+ 14482us) service_pool.cc:224] Handling call
0709 13:05:36.481986 (+ 22144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.493761 (+ 11775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.493793 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.541220 (+ 47427us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.972787 (+431567us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:36.972953 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.980986 (+  8033us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.755240 (+774254us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:37.822705 (+ 67465us) raft_consensus.cc:1685] finished
0709 13:05:37.822727 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.824753 (+  2026us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12232,"spinlock_wait_cycles":159785728,"thread_start_us":336316,"threads_started":1,"wal-append.queue_time_us":454426}
I20260709 13:05:37.801817 22437 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:05:37.821841 22359 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.833954 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 140199us
I20260709 13:05:37.838989 15814 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.813330 22455 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:37.835029 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.839973 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.840894 15842 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:37.841408 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:37.842424 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.830188 22435 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:37.802259 22461 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:05:37.823643 22456 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:37.827086 22354 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:37.837944 22290 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.644s	user 0.006s	sys 0.006s
I20260709 13:05:37.856319 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:37.832768 22457 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:37.830674 22029 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:37.815208 22227 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.857012 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.844122 22399 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:37.817441 22459 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:05:37.847793 22435 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:37.844067 22222 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:37.866863 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 60515.2us
I20260709 13:05:37.868067 15842 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
W20260709 13:05:37.857690 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 149510us
I20260709 13:05:37.877802 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.861095 22467 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:37.869587 22453 log.cc:927] Time spent T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:05:37.881368 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.879832 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:37.883417 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44512.8us
I20260709 13:05:37.829397 22264 consensus_queue.cc:237] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.846192 22359 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:37.869307 22389 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:37.862664 22290 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.891462 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:37.898710 15842 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.889575 15052 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:37.892448 15162 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:37.881690 22227 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:37.890219 22459 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:37.894194 15161 raft_consensus.cc:1275] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
W20260709 13:05:37.894814 22150 consensus_meta.cc:287] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.868s	user 0.012s	sys 0.001s
I20260709 13:05:37.892542 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:37.897769 15127 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:37.901997 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.903373 15161 pending_rounds.cc:85] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 14
I20260709 13:05:37.907234 15841 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:37.907459 15161 pending_rounds.cc:107] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:05:37.907768 15161 raft_consensus.cc:2889] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.907840 15162 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:37.906308 22470 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:37.895226 22118 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 13:05:37.882054 22463 raft_consensus.cc:2804] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:37.912772 15229 raft_consensus.cc:1275] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:05:37.880245 22289 consensus_queue.cc:1048] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:05:37.865164 22457 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.915881 15088 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:37.879635 22222 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:37.961054 15229 pending_rounds.cc:85] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 10
I20260709 13:05:37.963244 15229 pending_rounds.cc:107] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:05:37.963599 15229 raft_consensus.cc:2889] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.973923 15083 raft_consensus.cc:2410] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:05:37.910519 22399 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.911114 22470 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:37.922902 22458 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:37.908300 22464 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:37.900592 22389 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.961633 22461 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:37.998165 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 105704us
I20260709 13:05:37.972584 22437 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:37.889614 22467 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:37.967296 22468 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.005220 15128 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:37.970106 22462 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.009747 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 112058us
I20260709 13:05:38.017879 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.984545 22029 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:37.984234 22118 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.019179 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:37.999771 15086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1868a2755dea467c9c719191d9927234"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:05:38.016887 22182 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:38.021204 15086 raft_consensus.cc:606] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:37.908658 22354 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.022431 15848 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:38.023470 15086 raft_consensus.cc:614] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:37.996665 22470 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:38.022928 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.000125 22464 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.030614 15087 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:38.022071 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:38.033563 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.999559 22463 raft_consensus.cc:697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:38.002259 22112 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.011488 22437 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.035247 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.045288 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.047260 15081 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:38.047907 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.013710 22474 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:37.993752 22399 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.022632 22471 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.026690 22118 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.021756 22029 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.049867 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.052778 15087 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:38.016921 22359 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.024070 22462 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.035151 22464 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:38.049733 22437 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:38.064342 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 63017.6us
I20260709 13:05:38.065366 15202 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:38.104223 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:38.049907 22289 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:38.062235 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 328346us
I20260709 13:05:38.057026 22029 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.028453 22480 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:05:38.117235 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:38.022362 22468 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.049703 22249 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:05:38.121526 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.050179 22112 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.142136 15161 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:38.122562 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.116082 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.127038 15158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1679) took 1205 ms. Trace:
I20260709 13:05:38.127404 15842 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:38.130580 22408 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:05:38.135201 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.135427 22085 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.122401 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.145833 15082 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:38.042667 22478 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.077s
W20260709 13:05:38.139724 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.027886 22461 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.220260 15086 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:38.105351 22435 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.137248 22222 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.102767 22359 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.111232 22289 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:38.150161 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 120379us
I20260709 13:05:38.152896 15082 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:38.115880 22475 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:38.117321 22431 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:38.222256 15085 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:38.137131 22118 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.230654 15233 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:05:38.144526 15158 rpcz_store.cc:276] 0709 13:05:36.922080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.922260 (+   180us) service_pool.cc:224] Handling call
0709 13:05:36.922563 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.978600 (+ 56037us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:37.298848 (+320248us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.489787 (+190939us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:37.489875 (+    88us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:37.489941 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.126820 (+636879us) spinlock_profiling.cc:243] Waited 55.8 ms on lock 0x7b54003e0b54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:38.126866 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.126977 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":379262,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1797,"raft.queue_time_us":3439,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5424,"spinlock_wait_cycles":128279424,"thread_start_us":483353,"threads_started":2,"wal-append.queue_time_us":309141}
I20260709 13:05:38.060477 22474 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:38.232621 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 82915.4us
I20260709 13:05:38.235687 15081 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:05:38.261504 15081 pending_rounds.cc:85] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Aborting all ops after (but not including) 14
I20260709 13:05:38.217958 22290 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.242532 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.059597 22471 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.262980 15235 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:38.263221 15082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:05:38.065747 22477 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.163s
I20260709 13:05:38.264202 15082 raft_consensus.cc:606] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:38.264806 15082 raft_consensus.cc:614] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:38.246455 15128 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:38.248220 22491 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:38.249310 15083 raft_consensus.cc:1275] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:38.231402 15088 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:05:38.256500 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 153141us
I20260709 13:05:38.261797 15081 pending_rounds.cc:107] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:05:38.271164 15081 raft_consensus.cc:2889] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:38.244910 22289 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:38.276928 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.251205 22431 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.285375 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.257262 22381 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.285310 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.287806 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.289141 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.289458 15082 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:38.289816 15845 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 15.
I20260709 13:05:38.288690 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:38.288688 22381 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.293603 22381 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.264256 22222 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.304422 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.266849 22408 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.328639 22112 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.257843 22433 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.255054 22483 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:05:38.331385 22433 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.332240 22468 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:38.332625 22476 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:05:38.333899 22085 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.336045 22461 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.306473 22467 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:38.303640 22373 consensus_meta.cc:287] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.005s
I20260709 13:05:38.337682 22373 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.338477 15232 raft_consensus.cc:1240] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:38.340793 15839 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:38.340178 22373 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.295632 22431 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.343020 15154 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:38.346555 22471 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:38.336196 22362 consensus_meta.cc:287] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.004s
I20260709 13:05:38.353487 22362 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.355844 15079 raft_consensus.cc:2393] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:38.358922 22362 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.114987 22463 consensus_queue.cc:237] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.282002 22457 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.283489 22435 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:38.392753 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 108109us
I20260709 13:05:38.306629 22389 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.253671 22359 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.387259 22480 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.408800 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 144706us
I20260709 13:05:38.388736 22491 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.413221 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:38.386902 22325 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.413918 15237 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:38.391732 22461 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.403030 22462 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.372236 22408 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.325596 22222 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.424867 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:38.394654 22483 consensus_queue.cc:1048] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:38.395272 22467 consensus_queue.cc:1048] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:05:38.437878 22474 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.442524 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:38.443835 22443 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.444422 22443 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:38.445606 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1581) took 1361 ms. Trace:
I20260709 13:05:38.415005 22435 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:38.444784 22443 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.453480 15086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:05:38.420651 22491 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:38.456485 22319 consensus_meta.cc:287] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.000s
I20260709 13:05:38.455168 22408 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.445959 15229 rpcz_store.cc:276] 0709 13:05:37.084268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.098887 (+ 14619us) service_pool.cc:224] Handling call
0709 13:05:37.099146 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.964255 (+865109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.445399 (+481144us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b54000ad7d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:38.445550 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870418688}
I20260709 13:05:38.463650 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:38.459496 15086 raft_consensus.cc:606] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:38.459190 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:38.464321 15086 raft_consensus.cc:614] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:38.464550 15230 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.465342 15234 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:38.468441 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.426820 22461 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.472090 22461 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.418190 22480 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.475549 22480 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.476992 22435 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:38.469866 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 158812us
I20260709 13:05:38.484825 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:38.423220 22325 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.462586 22443 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:38.492004 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.506310 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:38.502178 22456 consensus_meta.cc:287] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.003s
I20260709 13:05:38.508991 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:38.509168 15230 raft_consensus.cc:2468] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:38.510017 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:38.516551 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.520754 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.521409 15842 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:38.523922 15840 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:38.530128 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.514516 22354 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:38.540021 22444 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.532297 22315 consensus_queue.cc:1059] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:05:38.531438 15841 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:38.530947 22498 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:38.519013 22462 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:38.551695 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.552011 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.21055e+06us
I20260709 13:05:38.519229 22465 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:38.550592 22322 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.561719 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:38.556037 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:38.523697 22495 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.567016 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:38.544203 22455 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.000s
I20260709 13:05:38.572060 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:38.527508 22290 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.573904 15201 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:38.571784 14979 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:38.578673 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.556684 22498 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:38.562295 22319 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.564920 22465 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.587798 15841 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:38.604812 15162 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:38.582154 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:38.564002 22444 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.587353 22462 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:38.606503 22462 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.515228 22456 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.582625 22455 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.606757 15201 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:38.607076 22462 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:38.608773 22455 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.575940 22264 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:38.606647 22263 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:38.612156 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:38.609395 22455 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:38.542922 22431 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:38.622525 22437 consensus_meta.cc:287] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 13:05:38.588581 22501 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:05:38.568789 22497 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:38.533785 22470 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.580938 22381 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:38.599437 22029 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.583513 22474 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:38.616885 22455 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.581703 22499 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.623427 22508 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.604357 21929 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.788s	user 0.006s	sys 0.006s
I20260709 13:05:38.610637 22465 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.650192 22433 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.661736 22433 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.663380 22289 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.663900 22433 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.664510 22433 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:38.612668 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:38.640756 22362 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:38.583351 22471 consensus_queue.cc:1059] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:05:38.635207 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.680166 22289 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.645042 22222 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.680964 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.616086 22263 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.682255 22263 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:38.585129 22290 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.576938 22495 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.602317 22498 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:38.666173 22470 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.666621 15163 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
W20260709 13:05:38.668958 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.610467 22319 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.683753 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.642606 22085 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.642869 22381 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.642742 22435 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.651721 22455 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:38.653817 22444 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.707990 22461 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.676333 22378 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.882s	user 0.009s	sys 0.003s
W20260709 13:05:38.692476 22366 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.043s	user 0.010s	sys 0.003s
I20260709 13:05:38.712744 22381 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.722524 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:38.636924 22437 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:38.721730 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 165291us
I20260709 13:05:38.660650 22462 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.683439 22433 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.755407 15128 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:38.719210 22491 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.759521 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:38.678622 21929 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.761816 15128 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:38.767395 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64374.7us
I20260709 13:05:38.674492 22508 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.717774 22085 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.672924 22499 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.719466 22435 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:38.721511 22378 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.724218 22461 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:38.807253 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.745586 22366 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.763907 22467 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:38.792896 22437 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:38.831656 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.799182 21942 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.834429 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 85007us
W20260709 13:05:38.832935 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.14013e+06us
W20260709 13:05:38.805367 22510 log.cc:927] Time spent T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:05:38.875062 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:38.765487 22289 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.787127 22335 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:38.872789 15847 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:38.762502 22470 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:38.786933 22492 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:05:38.874143 22444 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:38.891458 22290 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.927373 15842 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:38.892709 22499 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:38.893172 22264 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:38.894459 22465 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:38.896034 22480 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:38.896349 22378 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.897264 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.945904 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.897794 22085 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:38.898307 15157 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:38.898931 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.899622 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:38.900262 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.902791 22501 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:38.903785 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:38.906762 22466 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.907729 22469 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:38.908735 21929 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.910259 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.911235 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:38.913702 22433 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.913827 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:38.915490 22464 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.002s
I20260709 13:05:38.919088 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:38.922075 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.927341 22335 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.923882 22390 raft_consensus.cc:3055] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:38.916468 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1695) took 1031 ms. Trace:
I20260709 13:05:38.936419 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:38.941102 22456 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:38.977710 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.923246 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:38.964620 15088 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:38.978813 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:38.965364 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 61813us
I20260709 13:05:38.965881 15155 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:38.978649 15162 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:38.979489 15157 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.952462 15156 raft_consensus.cc:2368] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:38.953975 15158 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:38.977710 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.980450 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:38.976157 22485 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.867s	user 0.002s	sys 0.000s
I20260709 13:05:38.981925 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:38.984865 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.985805 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:38.986359 15159 raft_consensus.cc:2376] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19. Re-sending same reply.
I20260709 13:05:38.971558 15154 rpcz_store.cc:276] 0709 13:05:37.885197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.885397 (+   200us) service_pool.cc:224] Handling call
0709 13:05:37.893488 (+  8091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.343432 (+449944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.916138 (+572706us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:38.916387 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032957824}
I20260709 13:05:38.978917 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:38.924693 21942 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:38.992904 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 102193us
I20260709 13:05:38.954977 22501 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:38.993883 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:38.996795 15158 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:38.942987 22444 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.000533 15128 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:38.946322 22443 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.003260 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:38.971395 22390 raft_consensus.cc:740] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:39.050061 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:38.972335 22470 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:38.994369 22511 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:05:38.987179 22335 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:38.984491 22029 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.052619 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:38.992239 22354 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:38.992375 22253 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:39.055477 15085 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:38.952927 22480 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.061296 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.069098 15088 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:39.002427 22501 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:38.967234 22378 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:39.064853 15079 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:38.971269 22464 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.051975 22457 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:39.069335 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 75532.8us
I20260709 13:05:38.965337 22466 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.068418 22264 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.048372 22227 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:38.992599 22467 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 13:05:39.069939 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68290.6us
I20260709 13:05:39.073864 15157 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:39.012648 22459 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:05:39.057008 22492 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:05:39.072149 22472 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.135s
I20260709 13:05:39.207355 22465 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.207592 22472 consensus_queue.cc:1243] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:05:39.126259 22522 consensus_queue.cc:1048] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:05:39.067279 22354 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.210021 22354 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.125972 22462 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:39.133121 22322 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.559s	user 0.001s	sys 0.010s
I20260709 13:05:39.215842 22471 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.066118 22495 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.218359 22466 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:39.130635 22444 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:39.219355 22481 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.105s	user 0.000s	sys 0.002s
I20260709 13:05:39.131044 22390 consensus_queue.cc:260] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.216513 22471 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.221176 22390 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:39.222136 15086 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:39.222041 15847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59152 (request call id 1670) took 1381 ms. Trace:
I20260709 13:05:39.222704 15847 rpcz_store.cc:276] 0709 13:05:37.841044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.841177 (+   133us) service_pool.cc:224] Handling call
0709 13:05:39.221741 (+1380564us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540041c154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:39.221964 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370183040}
I20260709 13:05:39.189345 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3260) took 1059 ms. Trace:
W20260709 13:05:39.186163 22263 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:05:39.198029 15839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1696) took 1856 ms. Trace:
I20260709 13:05:39.224776 15083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:05:39.225340 15839 rpcz_store.cc:276] 0709 13:05:37.341658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.341822 (+   164us) service_pool.cc:224] Handling call
0709 13:05:37.347262 (+  5440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.341273 (+994011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.197680 (+856407us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b5400600254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:39.197943 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2284376448}
I20260709 13:05:39.226572 15083 raft_consensus.cc:606] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:39.196566 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.228355 15080 raft_consensus.cc:2368] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:39.167285 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.229588 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2754) took 1107 ms. Trace:
I20260709 13:05:39.076576 22029 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.230057 15163 rpcz_store.cc:276] 0709 13:05:38.121712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.122222 (+   510us) service_pool.cc:224] Handling call
0709 13:05:39.209489 (+1087267us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b540037e954. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:39.229529 (+ 20040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202788096}
I20260709 13:05:39.074064 15810 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:39.125080 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.133729 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:39.097720 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.181099 15088 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
W20260709 13:05:39.055954 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 124380us
I20260709 13:05:39.224398 15087 rpcz_store.cc:276] 0709 13:05:38.129622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.129850 (+   228us) service_pool.cc:224] Handling call
0709 13:05:38.130304 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.251517 (+121213us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:38.390934 (+139417us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.515084 (+124150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.515145 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:38.515188 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.189001 (+673813us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540014e454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:39.189070 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.189263 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15696,"prepare.queue_time_us":76908,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":21582,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":10099,"spinlock_wait_cycles":278051712,"thread_start_us":253063,"threads_started":2,"wal-append.queue_time_us":195716}
W20260709 13:05:39.110471 14983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 324) took 43151 ms (client timeout 15000 ms). Trace:
I20260709 13:05:39.132105 22443 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 25 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.111743 22325 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.091653 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.105346 14979 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 13, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.253521 15085 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 25: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
W20260709 13:05:39.256003 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.186017 22487 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.076s	user 0.004s	sys 0.000s
I20260709 13:05:39.269058 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.269625 15236 raft_consensus.cc:2376] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20. Re-sending same reply.
I20260709 13:05:39.269704 15230 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:05:39.270471 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.271250 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.259308 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:39.271972 15842 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:39.268355 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:39.272711 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.273301 15230 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:39.275365 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.276300 15235 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:39.276190 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.127728 22464 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.277891 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:39.275516 15237 raft_consensus.cc:1240] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:39.068209 22253 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:39.288311 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 214310us
I20260709 13:05:39.176829 22366 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.284475 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.182794 22353 raft_consensus.cc:3060] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.144306 22463 raft_consensus.cc:3055] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:39.219537 22289 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:39.106353 22473 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:39.149245 22290 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.292683 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1590) took 1063 ms. Trace:
I20260709 13:05:39.354472 15232 rpcz_store.cc:276] 0709 13:05:38.229438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.229645 (+   207us) service_pool.cc:224] Handling call
0709 13:05:38.230039 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.338924 (+108885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.292444 (+953520us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400276fd4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:39.292624 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248900864}
I20260709 13:05:39.293262 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:39.297711 15154 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:39.166159 22435 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.152998 22508 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.238140 22489 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:39.299839 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.101091 22085 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.225040 22263 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.300169 15158 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:39.305491 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:39.376415 15156 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:39.308759 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:39.377178 15159 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:39.308921 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.311157 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.378289 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.378533 15237 raft_consensus.cc:2410] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:39.122757 22461 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.311542 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.147347 21942 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.314561 15088 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:39.355003 22359 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.315340 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:39.388952 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:39.389932 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 74685.2us
I20260709 13:05:39.390532 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:39.390851 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:39.391355 15158 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:39.316313 15085 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:05:39.318152 15086 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:39.392853 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:39.319362 15083 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:39.315348 15202 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:39.395761 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 80504.8us
W20260709 13:05:39.330605 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 73808.7us
I20260709 13:05:39.399112 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:39.100164 22381 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.322204 14979 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: config changed from index 5 to 8, term changed from 12 to 14, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 12daa67457954a0cab27e32bf85fa115 (127.13.163.193), VOTER 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194), VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195), VOTER 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 14 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.367553 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.369596 15079 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:39.355567 22531 raft_consensus.cc:2804] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:39.190979 22480 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:39.370723 15051 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.503s (SENT)
W20260709 13:05:39.422508 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51822.9us
I20260709 13:05:39.082032 22264 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:39.374665 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 85598.8us
I20260709 13:05:39.371888 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.375418 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:39.394542 22529 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:39.379015 15235 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:39.318573 22463 raft_consensus.cc:740] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:39.355824 15847 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:39.382601 15128 leader_election.cc:400] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:39.447436 15128 leader_election.cc:403] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:05:39.363018 15163 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:39.447685 15085 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:39.350075 22473 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:39.422286 22474 consensus_meta.cc:287] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.000s
I20260709 13:05:39.366214 15842 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:39.075359 22227 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:39.387877 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.388499 15154 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:39.081604 22457 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.416271 15161 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 25: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:39.323549 22112 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:05:39.423695 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:39.371096 22489 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:39.431999 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:39.547658 15156 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:05:39.429157 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.435806 15229 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:39.257148 22029 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.432502 15232 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:39.432992 15088 raft_consensus.cc:1240] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:39.444973 15079 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:39.444902 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:39.479212 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.367453 22290 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.380040 22530 consensus_queue.cc:1048] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:05:39.448988 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.452589 15080 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:39.416685 22461 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:39.385452 22399 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.424259 22359 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:39.565352 22473 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:39.595474 15235 raft_consensus.cc:1240] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:39.382699 22085 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.444312 22431 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:39.436300 22466 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.587080 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:39.596176 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 143768us
I20260709 13:05:39.426249 22527 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:39.603579 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 204551us
I20260709 13:05:39.600423 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.441668 14979 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.634107 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:39.575246 22498 consensus_meta.cc:287] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.860s	user 0.008s	sys 0.002s
I20260709 13:05:39.627277 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.588469 22362 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.635280 15088 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:39.635577 22362 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.636211 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.636797 15088 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:39.637724 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.634428 15054 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:39.638671 22466 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.388214 22253 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.640144 22253 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.567324 22474 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:39.564957 22227 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.651764 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.539465 22150 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:05:39.662341 15229 raft_consensus.cc:3060] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:39.664994 14979 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.573447 22469 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:05:39.596153 22359 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.611654 22378 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:39.664208 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:39.664784 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.678761 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44721.6us
I20260709 13:05:39.450387 22443 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.160s
I20260709 13:05:39.572662 22456 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:05:39.669986 15232 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:39.638851 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.690887 15229 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:39.690196 14979 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:39.682017 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.677954 15237 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:39.692365 14979 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 16, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.651580 22531 raft_consensus.cc:697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:39.694898 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.704250 15232 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:39.254314 14983 rpcz_store.cc:269] 0709 13:04:55.958626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.960993 (+  2367us) service_pool.cc:224] Handling call
0709 13:04:56.070731 (+109738us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:39.110232 (+43039501us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:21.728708 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:21.733519 (+  4811us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:21.733550 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:05:21.733567 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:21.735118 (+  1551us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:21.735205 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:21.735242 (+    37us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:05:21.736604 (+  1362us) tablet.cc:701] Row locks acquired
0709 13:05:21.736620 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:05:21.736772 (+   152us) write_op.cc:270] Start()
0709 13:05:21.742530 (+  5758us) write_op.cc:276] Timestamp: P: 1783602321736719 usec, L: 0
0709 13:05:21.742554 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:05:21.812497 (+ 69943us) log.cc:844] Serialized 7884 byte log entry
0709 13:05:22.674401 (+861904us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:22.694908 (+ 20507us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18003474a0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:23.803453 (+1108545us) op_driver.cc:464] REPLICATION: finished
0709 13:05:24.373142 (+569689us) write_op.cc:301] APPLY: starting
0709 13:05:24.373295 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:25.155279 (+781984us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:25.155332 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:25.443395 (+288063us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:25.443829 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:05:25.443900 (+    71us) write_op.cc:312] APPLY: finished
0709 13:05:25.444479 (+   579us) log.cc:844] Serialized 180 byte log entry
0709 13:05:25.762659 (+318180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:25.824097 (+ 61438us) write_op.cc:454] Released schema lock
0709 13:05:25.840327 (+ 16230us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:25.881237 (+ 40910us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:25.881299 (+    62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28941,"reported_tablets":45,"spinlock_wait_cycles":15958400,"child_traces":[["op",{"apply.queue_time_us":552647,"apply.run_cpu_time_us":78940,"apply.run_wall_time_us":1508030,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87847,"num_ops":20,"prepare.queue_time_us":131352,"prepare.run_cpu_time_us":42754,"prepare.run_wall_time_us":594457,"raft.queue_time_us":1017585,"raft.run_cpu_time_us":73042,"raft.run_wall_time_us":557728,"replication_time_us":2030951,"spinlock_wait_cycles":10757376,"thread_start_us":1809593,"threads_started":5,"wal-append.queue_time_us":903258,"wal-append.run_cpu_time_us":110025,"wal-append.run_wall_time_us":3393350}]]}
I20260709 13:05:39.666728 22461 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:39.657493 22466 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.611460 22483 consensus_queue.cc:1059] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:05:39.705265 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:39.727988 15232 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 15: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:39.588621 22325 consensus_queue.cc:1059] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
W20260709 13:05:39.677078 22455 consensus_meta.cc:287] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.006s
I20260709 13:05:39.637778 22362 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.677525 22359 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:39.672976 22527 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:39.724645 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:39.690148 22408 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:39.728220 14979 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 16, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.743018 14979 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 14, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 14 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:39.655788 22498 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.702267 15237 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:39.773592 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.789322 15237 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:05:39.814879 15158 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:39.653832 22463 consensus_queue.cc:260] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.825644 15841 raft_consensus.cc:1275] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:39.823874 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:39.827872 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:39.783581 22530 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.755389 22461 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:39.829644 15842 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c"
dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
mode: GRACEFUL
new_leader_uuid: "3d45101e0851484690b4f03a4dcdb63d"
 from {username='slave'} at 127.0.0.1:35386
I20260709 13:05:39.831398 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:39.833267 15842 raft_consensus.cc:606] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:39.833647 15842 raft_consensus.cc:614] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:39.833997 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.837625 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:39.838954 15810 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:39.672122 22489 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.847290 14979 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 17, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.835820 22463 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:39.825989 22264 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.844172 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:39.834704 15085 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:39.829706 22517 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:39.842371 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 180883us
I20260709 13:05:39.838198 22389 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.850042 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1730) took 1089 ms. Trace:
I20260709 13:05:39.869215 14979 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 15, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 15 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.864141 15847 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:39.872222 15848 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:39.840710 22466 raft_consensus.cc:3055] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:39.838430 22472 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.759047 22455 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.846292 22381 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.887533 14979 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.861164 22378 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:39.898226 14979 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 18, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:39.882884 22534 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 13:05:39.871186 22408 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:39.877822 22335 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.004s
I20260709 13:05:39.858809 22457 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.852614 22037 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:05:39.908659 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.020s	sys 0.026s
W20260709 13:05:39.926726 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.021s	sys 0.026s
I20260709 13:05:39.841826 22530 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.876209 22466 raft_consensus.cc:740] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:39.898149 22547 log.cc:927] Time spent T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:05:39.858335 22352 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:05:39.875700 22546 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:39.882660 22489 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.882416 22483 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:39.918470 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 79619.5us
I20260709 13:05:39.923347 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:39.923669 14979 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 17, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:39.920976 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:39.916567 22536 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 13:05:39.916110 22488 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.800s	user 0.005s	sys 0.000s
I20260709 13:05:39.903371 22378 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.936054 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:39.902153 22290 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.906662 22029 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.916797 22408 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.850843 22531 consensus_queue.cc:237] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:39.954500 15847 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:39.914238 22085 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.886853 22389 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.012300 22455 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.961241 14979 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.957443 22037 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.971799 22457 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:39.877530 22517 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:39.998039 22543 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:05:40.087031 22399 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:40.087734 22399 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:40.084580 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.964520 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:40.090297 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 125849us
I20260709 13:05:39.884735 22472 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.101784 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:39.991778 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 70958.5us
I20260709 13:05:39.955235 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:39.999832 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:40.016320 22353 consensus_meta.cc:287] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.005s
I20260709 13:05:39.923946 22461 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:40.105803 15087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.000s	sys 0.007s
I20260709 13:05:40.000553 22408 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.935691 22381 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:05:39.979427 22319 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:39.976687 22530 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.053289 22290 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.901891 22527 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:40.061020 15079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d"
dest_uuid: "12daa67457954a0cab27e32bf85fa115"
mode: GRACEFUL
new_leader_uuid: "4b633b84351f40de9437e3d532fe6fa7"
 from {username='slave'} at 127.0.0.1:34666
I20260709 13:05:40.128876 15079 raft_consensus.cc:606] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.129269 15079 raft_consensus.cc:614] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:40.066107 22517 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.057225 22037 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.073035 22498 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:40.077704 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3267) took 1530 ms. Trace:
I20260709 13:05:40.153285 22037 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:40.152407 15081 rpcz_store.cc:276] 0709 13:05:38.547775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.309488 (+761713us) service_pool.cc:224] Handling call
0709 13:05:39.317250 (+  7762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.317503 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:39.446151 (+128648us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.075415 (+629264us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b180036aae0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219
0709 13:05:40.076303 (+   888us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b180036aae0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d438b858f 00007f0d3c8f7571 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd
0709 13:05:40.077151 (+   848us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.077340 (+   189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.077390 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:40.077438 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.077505 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.077631 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":408575,"prepare.queue_time_us":691509,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":4641,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":12134,"spinlock_wait_cycles":72597120,"thread_start_us":310269,"threads_started":2,"wal-append.queue_time_us":363401}
I20260709 13:05:40.155463 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:39.870473 15083 rpcz_store.cc:276] 0709 13:05:38.760899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.395811 (+634912us) service_pool.cc:224] Handling call
0709 13:05:39.437439 (+ 41628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.437615 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:39.677358 (+239743us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.849411 (+172053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:39.849520 (+   109us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:39.849595 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.849727 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.849950 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1962,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":8211,"thread_start_us":267173,"threads_started":2}
I20260709 13:05:40.157874 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:05:40.081961 22528 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 13:05:40.054651 22466 consensus_queue.cc:260] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:39.995052 22378 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:40.159487 22520 raft_consensus.cc:3055] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:40.159741 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:40.159914 22520 raft_consensus.cc:740] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:40.163080 22542 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.264s	user 0.005s	sys 0.000s
W20260709 13:05:40.163753 22545 log.cc:927] Time spent T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.267s	user 0.000s	sys 0.005s
W20260709 13:05:40.164428 22540 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.348s	user 0.006s	sys 0.000s
I20260709 13:05:39.929392 22335 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:40.166464 22335 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:39.960058 22546 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.004720 22389 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.111337 15846 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:40.110324 22362 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:40.119758 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:40.120121 22539 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:05:40.108930 22444 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
W20260709 13:05:40.120586 22473 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.005s
W20260709 13:05:40.210801 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.210609 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.204823 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:40.120708 22515 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 1.227s	user 0.006s	sys 0.000s
I20260709 13:05:40.122787 22359 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:40.124243 22354 consensus_meta.cc:287] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.914s	user 0.006s	sys 0.002s
I20260709 13:05:40.096429 22457 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:40.133996 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2598) took 1230 ms. Trace:
I20260709 13:05:40.135865 22399 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.239872 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.240257 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:40.240900 15083 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:40.133823 22290 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.143119 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:40.146095 22530 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:40.147586 22544 log.cc:927] Time spent T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.251s	user 0.000s	sys 0.005s
W20260709 13:05:40.148033 22537 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.119s	user 0.000s	sys 0.005s
I20260709 13:05:40.125349 22227 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:40.156214 15081 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:40.106686 15087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.000s	sys 0.008s
I20260709 13:05:40.108681 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:40.272532 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 163928us
I20260709 13:05:40.179175 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.191637 15842 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:05:40.194977 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.016s
I20260709 13:05:40.212476 15845 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:40.235240 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:40.225046 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 846569us
I20260709 13:05:40.239531 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.138270 15079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:40.236307 15085 rpcz_store.cc:276] 0709 13:05:38.903274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.872656 (+969382us) service_pool.cc:224] Handling call
0709 13:05:39.898899 (+ 26243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.899125 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:40.077851 (+178726us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.133541 (+ 55690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.133612 (+    71us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:40.133687 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.133774 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.133928 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181531,"prepare.queue_time_us":179144,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":4409,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":52284,"threads_started":1}
W20260709 13:05:40.237053 22528 log.cc:927] Time spent T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.078s	user 0.008s	sys 0.000s
I20260709 13:05:40.258347 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.307210 15841 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:40.194801 22370 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.267025 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3262) took 1838 ms. Trace:
I20260709 13:05:40.275399 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:40.275957 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.003s	sys 0.016s
W20260709 13:05:40.311094 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 651297us
I20260709 13:05:40.315780 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.316438 15845 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:40.317325 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.277390 15844 raft_consensus.cc:2376] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17. Re-sending same reply.
I20260709 13:05:40.253445 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.279019 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.279198 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:40.318804 15085 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:40.279639 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.318917 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.320098 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:40.280297 22552 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.304s	user 0.000s	sys 0.005s
W20260709 13:05:40.280666 22561 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.094s	user 0.000s	sys 0.004s
I20260709 13:05:40.331910 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.281471 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.332772 15236 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:40.332917 15162 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 13:05:40.282269 22554 log.cc:927] Time spent T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.099s	user 0.002s	sys 0.007s
I20260709 13:05:40.291823 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.292012 22383 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.291903 22429 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:40.265652 22480 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:40.294605 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.294845 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.295778 22470 consensus_queue.cc:1059] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:40.298306 22546 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.299448 22499 consensus_queue.cc:1059] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:40.299852 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 120820us
I20260709 13:05:40.300321 21942 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.300865 22029 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.320369 22037 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:40.302640 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.304673 15839 raft_consensus.cc:2368] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:40.238293 22444 consensus_queue.cc:1243] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:40.310345 15840 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
W20260709 13:05:40.321421 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 68097us
I20260709 13:05:40.318827 15848 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 15: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:40.328197 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.308593 15082 rpcz_store.cc:276] 0709 13:05:38.429047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.240281 (+811234us) service_pool.cc:224] Handling call
0709 13:05:39.240680 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.241127 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.241169 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.241742 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.443970 (+202228us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:39.444028 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.444132 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.266723 (+822591us) raft_consensus.cc:1685] finished
0709 13:05:40.266756 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.266945 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23052,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":92013,"threads_started":1,"wal-append.queue_time_us":343651}
I20260709 13:05:40.330531 15079 raft_consensus.cc:2468] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:40.239991 22431 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.334964 15081 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:05:40.334964 15162 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:40.338119 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.456552 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2599) took 1550 ms. Trace:
I20260709 13:05:40.340693 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.456782 15163 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:40.261543 22457 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:40.366063 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.369671 15158 raft_consensus.cc:1240] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:40.293898 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.373597 15155 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:05:40.237900 22473 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.375489 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:40.468356 15236 raft_consensus.cc:2376] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17. Re-sending same reply.
I20260709 13:05:40.300349 22317 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:40.379402 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:40.320642 22437 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.242362 22354 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.301404 22335 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.483598 22335 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.485001 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.486181 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:40.486352 22335 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.486670 15231 raft_consensus.cc:2376] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15. Re-sending same reply.
I20260709 13:05:40.487571 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:40.491559 22489 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.536s	user 0.004s	sys 0.004s
I20260709 13:05:40.493528 22263 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:40.496497 22570 log.cc:927] Time spent T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 13:05:40.497987 22502 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:40.383461 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2585) took 2102 ms. Trace:
I20260709 13:05:40.498940 15084 rpcz_store.cc:276] 0709 13:05:38.281234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.281465 (+   231us) service_pool.cc:224] Handling call
0709 13:05:38.307156 (+ 25691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.324691 (+ 17535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.324735 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:38.325524 (+   789us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.165799 (+840275us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b18001dcf80. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:39.537538 (+371739us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:39.537577 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.537644 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.366044 (+828400us) raft_consensus.cc:1685] finished
0709 13:05:40.366070 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.383386 (+ 17316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2536,"raft.queue_time_us":18563,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":19258,"spinlock_wait_cycles":65655168,"thread_start_us":371565,"threads_started":1,"wal-append.queue_time_us":1551497}
I20260709 13:05:40.382792 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.501822 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.502516 15156 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:40.502825 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.329514 22433 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:40.503818 22433 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.504273 22433 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:40.504376 22472 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.505863 22227 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.507154 22359 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.508462 22290 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.396747 21942 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.509101 22437 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.428275 22037 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.511193 22437 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:40.511502 22501 consensus_meta.cc:287] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.431s	user 0.012s	sys 0.000s
I20260709 13:05:40.512054 22037 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:40.372848 22383 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:40.513468 22463 consensus_meta.cc:287] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.003s
I20260709 13:05:40.514411 22383 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:40.515210 22390 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.294s	user 0.009s	sys 0.003s
I20260709 13:05:40.515753 21942 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.516018 15088 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:40.517418 22571 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:40.518761 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2772) took 1130 ms. Trace:
I20260709 13:05:40.518689 22378 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.519347 15157 rpcz_store.cc:276] 0709 13:05:39.388095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.388510 (+   415us) service_pool.cc:224] Handling call
0709 13:05:39.401589 (+ 13079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.169594 (+768005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.169623 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.170093 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.262911 (+ 92818us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:40.262953 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.263215 (+   262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.518525 (+255310us) raft_consensus.cc:1685] finished
0709 13:05:40.518550 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.518697 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":767553,"raft.queue_time_us":806557,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":15685,"thread_start_us":92397,"threads_started":1,"wal-append.queue_time_us":171345}
I20260709 13:05:40.521018 15847 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:40.426797 22029 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.523672 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.524354 15232 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:40.525246 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:40.526278 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.526811 22473 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.527279 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.527513 15843 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:40.528765 15232 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:40.392450 15841 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:40.336784 22394 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.393910 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1728) took 1640 ms. Trace:
I20260709 13:05:40.531543 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.197333 22520 consensus_queue.cc:260] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.335481 22085 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:40.541971 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 162781us
I20260709 13:05:40.395563 15233 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:40.270675 22530 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.432235 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.397502 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.345880 22319 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:05:40.437995 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.455754 15085 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:40.552598 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.342938 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.457828 15081 rpcz_store.cc:276] 0709 13:05:38.906423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.328127 (+1421704us) service_pool.cc:224] Handling call
0709 13:05:40.334685 (+  6558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.456270 (+121585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.456485 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.453150 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:40.460317 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.555785 15230 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:40.465899 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.461768 15159 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:40.398694 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.556288 15840 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:05:40.556345 15158 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:40.556823 15160 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 15.
I20260709 13:05:40.467088 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.344807 22533 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 13:05:40.433604 22222 raft_consensus.cc:3055] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:40.408531 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.473760 15162 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:40.474361 15845 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:40.478484 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
W20260709 13:05:40.479851 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 88126us
I20260709 13:05:40.480486 22315 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.481171 22565 consensus_queue.cc:1059] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:05:40.500643 22471 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.532096 15154 raft_consensus.cc:1275] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:40.532745 15839 raft_consensus.cc:1240] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:40.511926 22501 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.488529 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.532816 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:40.534188 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:40.530633 15086 rpcz_store.cc:276] 0709 13:05:38.753884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.394678 (+640794us) service_pool.cc:224] Handling call
0709 13:05:39.395952 (+  1274us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:39.430660 (+ 34708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.430690 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:39.431395 (+   705us) log.cc:844] Serialized 78 byte log entry
0709 13:05:39.643558 (+212163us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:39.830416 (+186858us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.077377 (+246961us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.077576 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.077660 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.393710 (+316050us) raft_consensus.cc:1685] finished
0709 13:05:40.393727 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.393858 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":261676,"prepare.queue_time_us":666375,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":31308,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":24060,"thread_start_us":363129,"threads_started":2,"wal-append.queue_time_us":293164}
I20260709 13:05:40.549283 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:40.627684 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 78480.6us
I20260709 13:05:40.437089 22431 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.550908 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.339957 15083 raft_consensus.cc:2468] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
W20260709 13:05:40.554864 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101798us
W20260709 13:05:40.629680 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 221237us
I20260709 13:05:40.631158 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:40.336370 22370 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.550581 22394 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.634477 15086 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:40.559625 15233 raft_consensus.cc:1240] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:40.547582 21942 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:40.559926 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 122040us
I20260709 13:05:40.601502 22354 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.382987 22429 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.651343 22557 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:40.568856 15163 raft_consensus.cc:2393] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:40.652091 22085 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:40.652546 22085 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.546828 22325 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:40.532811 22029 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.652907 22085 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:40.636612 22383 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:40.654429 22383 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.654829 22383 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:40.621213 22315 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.469403 22475 consensus_queue.cc:1048] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.132s
I20260709 13:05:40.548147 22491 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:40.632450 15127 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:40.657826 15127 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:40.658329 22315 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.658809 15127 leader_election.cc:400] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:40.658991 22557 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:40.659540 22557 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:40.555245 15842 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:40.661417 22319 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:40.661957 22319 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:40.662855 22319 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.660372 22557 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.664791 22480 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:40.666683 22557 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:40.666733 22480 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.667717 22480 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:40.668458 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:40.637032 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 158628us
I20260709 13:05:40.668963 22489 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:40.665460 22378 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.669534 22489 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:40.601773 22150 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:40.670701 22150 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:40.640792 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 108035us
I20260709 13:05:40.672084 22317 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.672272 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:40.644649 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.674649 22353 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:40.675040 22353 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.675370 22353 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:40.675611 22431 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.676239 22431 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:40.676622 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.637758 15231 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:40.631654 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.678658 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.679464 15840 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
W20260709 13:05:40.679533 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 47947.4us
I20260709 13:05:40.680472 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.656052 22469 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:40.692226 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.670531 22489 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.675276 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.698995 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.608002 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.615547 22335 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:05:40.682693 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 229141us
I20260709 13:05:40.682660 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.614812 22222 raft_consensus.cc:740] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:40.693162 15840 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:40.629786 22564 raft_consensus.cc:3055] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:40.560341 22530 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:40.690925 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:40.648984 22471 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:40.702265 15203 leader_election.cc:400] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:40.714164 15203 leader_election.cc:403] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:40.704829 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.715337 15840 raft_consensus.cc:2393] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:40.713338 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36620 (request call id 9) took 1277 ms. Trace:
I20260709 13:05:40.715658 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.716043 15083 rpcz_store.cc:276] 0709 13:05:39.423321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.640343 (+1217022us) service_pool.cc:224] Handling call
0709 13:05:40.700576 (+ 60233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.716540 15844 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:40.717994 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.719664 15051 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.720764 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.625334 22499 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:05:40.731950 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3261) took 2303 ms. Trace:
I20260709 13:05:40.734028 15080 rpcz_store.cc:276] 0709 13:05:38.428408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.237995 (+809587us) service_pool.cc:224] Handling call
0709 13:05:39.238373 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.586463 (+348090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.731715 (+1145252us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002373d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:40.731898 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478053760}
W20260709 13:05:40.735916 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45109.3us
I20260709 13:05:40.736605 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.737424 15840 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:40.547019 22457 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.672665 22317 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.738875 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.698235 22150 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:40.698753 22354 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:40.742288 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:40.700769 22335 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:40.771903 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:40.701220 22443 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:40.776055 22394 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.776715 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.778337 22317 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:40.772981 14972 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12daa67457954a0cab27e32bf85fa115 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:40.782799 15231 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:40.783840 22335 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:40.785130 22335 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:40.787428 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:40.789132 15080 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:40.761874 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.796257 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:40.796478 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.796927 15234 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:40.797940 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:40.799537 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:40.757328 22431 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:40.739053 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:40.769479 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.805645 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:40.714368 22475 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:40.764236 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:40.714816 22564 raft_consensus.cc:740] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:40.741155 22444 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:05:40.739316 22471 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:40.768710 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:40.814527 15162 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:40.801331 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2600) took 1887 ms. Trace:
I20260709 13:05:40.760006 22455 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:40.703800 22290 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 13:05:40.796083 22362 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.001s
I20260709 13:05:40.811403 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:40.812134 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.809762 15840 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:40.847409 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:40.847728 15127 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.875s (SENT)
I20260709 13:05:40.787933 22433 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.804374 22317 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:40.912420 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:40.771371 22435 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:05:40.821722 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:40.821410 22222 consensus_queue.cc:260] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:40.804517 22390 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:40.822110 22569 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:40.822379 22394 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.822906 22564 consensus_queue.cc:260] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:40.804528 22408 consensus_meta.cc:287] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.000s
I20260709 13:05:40.960150 22444 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:40.869385 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 58029.5us
W20260709 13:05:40.872334 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.877032 15126 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.749s (SENT)
W20260709 13:05:41.018919 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 141937us
W20260709 13:05:41.019651 15126 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.755s (SENT)
I20260709 13:05:41.020114 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:40.848616 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:40.912075 15231 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:05:40.817020 22150 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:40.815289 15085 rpcz_store.cc:276] 0709 13:05:38.913596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.560381 (+1646785us) service_pool.cc:224] Handling call
0709 13:05:40.560665 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.560890 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:40.561170 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.800890 (+239720us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.800959 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:40.801017 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.801106 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.801261 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":459,"prepare.queue_time_us":560,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":4350,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5928,"thread_start_us":238813,"threads_started":1}
I20260709 13:05:40.808136 22381 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:40.974334 22464 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.824450 22443 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:40.993968 15839 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:40.828187 22475 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:40.826597 22029 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:40.825863 15234 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:40.915414 22362 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:41.025315 15052 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:41.026835 15088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:41.134485 15088 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
W20260709 13:05:41.137858 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 112628us
W20260709 13:05:41.028115 15126 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.875s (SENT)
I20260709 13:05:41.139827 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:40.960631 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 124313us
I20260709 13:05:40.993285 22569 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:41.031404 22575 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:41.084506 22480 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.010807 22437 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.111999 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:41.169920 15231 raft_consensus.cc:2410] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:41.112247 22029 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.166716 22359 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:41.188627 22029 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:41.190095 22353 consensus_meta.cc:287] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.002s
I20260709 13:05:41.113816 22501 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:41.190660 22353 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.114631 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:41.191957 15233 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.136694 22471 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.028894 22408 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.197356 22408 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:41.119468 22085 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.199988 22085 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.130182 22433 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:40.959913 22317 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:41.141026 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 112963us
I20260709 13:05:41.201187 22317 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:41.173194 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:41.144028 15237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.042s	sys 0.053s
W20260709 13:05:41.208396 15237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.042s	sys 0.053s
I20260709 13:05:41.028683 22444 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.209343 22456 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:41.015970 22394 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.209839 22444 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:41.104954 22564 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.209443 15237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1614) took 1334 ms. Trace:
I20260709 13:05:41.214129 15237 rpcz_store.cc:276] 0709 13:05:39.874571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.528813 (+654242us) service_pool.cc:224] Handling call
0709 13:05:40.529076 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.529276 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:40.529549 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.208975 (+679426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:41.209049 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:41.209122 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.209220 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.209371 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5649,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1938,"raft.queue_time_us":2600,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":679025,"threads_started":1}
I20260709 13:05:41.105885 22464 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.159072 22354 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:41.166443 22477 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:41.185813 22437 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.105396 22443 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.167150 22569 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:41.185940 22037 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:41.180231 22557 consensus_meta.cc:287] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.000s
I20260709 13:05:41.233608 15162 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:41.234905 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:41.253762 15847 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.237036 22477 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.243196 15233 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
W20260709 13:05:41.245834 15232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.035s	sys 0.072s
W20260709 13:05:41.257774 15232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.036s	sys 0.072s
I20260709 13:05:41.258862 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2533) took 1252 ms. Trace:
W20260709 13:05:41.246001 15230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.007s	sys 0.026s
I20260709 13:05:41.239523 22443 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:41.271150 15230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.008s	sys 0.027s
I20260709 13:05:41.277576 15847 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:41.246670 22037 consensus_queue.cc:1243] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:05:41.280432 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.292418 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:41.279072 15202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.004s	sys 0.016s
W20260709 13:05:41.297502 15202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.004s	sys 0.017s
I20260709 13:05:41.297523 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:41.306267 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.310549 22437 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.311180 22480 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:41.312100 22383 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.002s
I20260709 13:05:41.312641 22383 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.293180 22565 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18.
I20260709 13:05:41.301489 22085 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:41.299772 22353 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:41.326090 15159 raft_consensus.cc:2393] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 21: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:41.315032 22383 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.266957 22477 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.317294 22394 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.317670 22464 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.327245 22557 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:05:41.355906 22085 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:41.383095 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 215902us
I20260709 13:05:41.381820 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:41.385195 15162 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:41.396061 15162 raft_consensus.cc:2468] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:41.373118 22317 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:41.381279 22568 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.988s	user 0.005s	sys 0.000s
I20260709 13:05:41.381882 22359 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:41.428596 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.008s	sys 0.030s
W20260709 13:05:41.569650 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.009s	sys 0.030s
W20260709 13:05:41.570353 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 831305us
I20260709 13:05:41.429301 15202 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:41.443414 15082 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:41.429816 15126 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.112s (SENT) [suppressed 25 similar messages]
I20260709 13:05:41.379856 22444 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.534471 22461 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:41.430756 22464 consensus_queue.cc:1059] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.106s
W20260709 13:05:41.545408 22335 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.760s	user 0.010s	sys 0.001s
I20260709 13:05:41.552668 22085 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.554584 22477 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.556926 15230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1620) took 1461 ms. Trace:
I20260709 13:05:41.587772 15230 rpcz_store.cc:276] 0709 13:05:40.095152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.558945 (+463793us) service_pool.cc:224] Handling call
0709 13:05:40.559266 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.563502 (+  4236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.563536 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.563997 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.291294 (+727297us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:41.291355 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.291457 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.556599 (+265142us) raft_consensus.cc:1685] finished
0709 13:05:41.556628 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.556856 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3890,"raft.queue_time_us":41347,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":20098,"thread_start_us":726941,"threads_started":1,"wal-append.queue_time_us":834002}
I20260709 13:05:41.558601 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:41.398285 22473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.012s	sys 0.026s
W20260709 13:05:41.563535 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.003s	sys 0.010s
W20260709 13:05:41.618603 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.003s	sys 0.011s
W20260709 13:05:41.566615 15229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.000s	sys 0.016s
I20260709 13:05:41.427826 22475 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.619379 15229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.000s	sys 0.016s
I20260709 13:05:41.619621 22475 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:41.620188 22583 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
W20260709 13:05:41.621357 22473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.013s	sys 0.027s
I20260709 13:05:41.622080 22475 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:41.426985 22315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.057s	sys 0.078s
W20260709 13:05:41.622712 22315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.088s	sys 0.144s
I20260709 13:05:41.622654 22263 consensus_queue.cc:1059] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 13:05:41.624426 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 195187us
I20260709 13:05:41.625710 22289 raft_consensus.cc:3055] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:41.626139 22289 raft_consensus.cc:740] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:41.627028 22289 consensus_queue.cc:260] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.627923 22444 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.629076 22359 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.631716 22370 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:41.635942 22317 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.637365 22589 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:41.587447 22461 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.643646 22461 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:41.583823 22580 log.cc:927] Time spent T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:05:41.567716 22569 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.647042 22569 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:41.647850 22520 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:41.649343 22549 raft_consensus.cc:3055] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:05:41.649854 22549 raft_consensus.cc:740] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:41.650925 22549 consensus_queue.cc:260] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.652393 22549 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:41.608790 22431 consensus_meta.cc:287] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.796s	user 0.009s	sys 0.002s
I20260709 13:05:41.640502 22443 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:41.660574 22579 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.290s	user 0.000s	sys 0.004s
I20260709 13:05:41.682438 22464 consensus_queue.cc:1243] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:41.682945 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3278) took 2106 ms. Trace:
I20260709 13:05:41.683416 15082 rpcz_store.cc:276] 0709 13:05:39.576122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.647093 (+1070971us) service_pool.cc:224] Handling call
0709 13:05:40.647476 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.583231 (+935755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.682689 (+ 99458us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540034a154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:41.682875 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801907584}
I20260709 13:05:41.686259 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:41.666333 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:41.691956 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:41.572523 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 142730us
I20260709 13:05:41.559878 22354 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:41.693619 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 135107us
I20260709 13:05:41.583652 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:41.616804 15088 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:41.694968 15839 raft_consensus.cc:2393] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:41.684639 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:41.696260 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:41.696691 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:41.677460 22429 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.699172 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:41.700441 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:41.700539 15082 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
W20260709 13:05:41.700613 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 117058us
I20260709 13:05:41.701673 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:41.701750 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:41.702319 15082 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:41.722779 15080 raft_consensus.cc:1240] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:41.711102 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:41.725001 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1738) took 2040 ms. Trace:
I20260709 13:05:41.725602 15080 rpcz_store.cc:276] 0709 13:05:39.684964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.722263 (+2037299us) service_pool.cc:224] Handling call
0709 13:05:41.722600 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.724644 (+  2044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.724926 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:41.727129 15203 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:41.672231 22466 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.430580 22383 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.792021 15082 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:41.804814 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:41.755496 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.808272 15082 raft_consensus.cc:1240] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:41.809278 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1743) took 1701 ms. Trace:
I20260709 13:05:41.810328 15082 rpcz_store.cc:276] 0709 13:05:40.107438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.807572 (+1700134us) service_pool.cc:224] Handling call
0709 13:05:41.807984 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.808970 (+   986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.809214 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:41.712013 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:41.842823 15162 raft_consensus.cc:2393] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:41.843382 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 313266us
I20260709 13:05:41.663152 22431 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:41.689668 22535 raft_consensus.cc:3055] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:41.704723 22569 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:41.849215 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:41.853999 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.962441 15161 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:41.808326 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 89510.2us
I20260709 13:05:41.704317 22394 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:41.710067 22464 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.972497 22464 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.975538 22443 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:41.787601 22591 log.cc:927] Time spent T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:05:41.705873 22264 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:41.856617 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:41.849869 22533 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:41.940341 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 385962us
I20260709 13:05:41.940912 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:41.724725 22564 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:41.794588 22263 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.188s
I20260709 13:05:41.847960 22589 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.142s
I20260709 13:05:42.005055 22585 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:42.005607 22585 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:41.989666 22569 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.855762 22383 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.010555 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1722) took 1538 ms. Trace:
I20260709 13:05:42.010931 22383 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.977092 22394 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:42.011128 15154 rpcz_store.cc:276] 0709 13:05:40.471636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.503767 (+ 32131us) service_pool.cc:224] Handling call
0709 13:05:40.504008 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.624657 (+120649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.010270 (+1385613us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54000866d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:42.010486 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63916800}
I20260709 13:05:42.011962 22394 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.985360 22431 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:41.985314 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 128782us
I20260709 13:05:42.014459 15813 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:41.993346 22533 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.017462 22523 raft_consensus.cc:2804] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:42.018129 22523 raft_consensus.cc:697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:42.019348 22533 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.019068 22523 consensus_queue.cc:237] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:41.993358 22461 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:41.710480 22317 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:42.020427 22461 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.021232 22317 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.022712 22222 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:42.023059 22461 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:41.993470 22443 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:42.024139 22443 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.026350 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.027172 15084 raft_consensus.cc:2410] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:42.027613 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40400 (request call id 2604) took 1623 ms. Trace:
I20260709 13:05:42.027882 15084 rpcz_store.cc:276] 0709 13:05:40.403875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.026039 (+1622164us) service_pool.cc:224] Handling call
0709 13:05:42.027560 (+  1521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:41.699251 22354 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.030354 22354 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.032284 15236 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:42.032922 22557 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.691956 22475 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:42.033823 15236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43970 (request call id 2521) took 2119 ms (client timeout 1913 ms). Trace:
I20260709 13:05:42.034336 22475 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:42.034932 15236 rpcz_store.cc:269] 0709 13:05:39.914208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.479273 (+565065us) service_pool.cc:224] Handling call
0709 13:05:42.033635 (+1554362us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54000582d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.033747 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3401433344}
I20260709 13:05:41.857980 22535 raft_consensus.cc:740] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:42.041527 22535 consensus_queue.cc:260] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.042825 22535 raft_consensus.cc:3060] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:41.991889 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:42.046914 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55088.2us
I20260709 13:05:41.794839 22260 consensus_queue.cc:1059] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.187s
I20260709 13:05:42.049953 22260 raft_consensus.cc:3055] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:42.050413 22260 raft_consensus.cc:740] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:42.051456 22260 consensus_queue.cc:260] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:41.993010 22564 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.053851 22564 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:41.852708 22444 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:42.033406 22557 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.058027 15157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1725) took 1580 ms. Trace:
I20260709 13:05:42.063645 15157 rpcz_store.cc:276] 0709 13:05:40.477169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.531257 (+ 54088us) service_pool.cc:224] Handling call
0709 13:05:40.542796 (+ 11539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.543121 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.543150 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.553423 (+ 10273us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.619062 (+1065639us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:41.619123 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.619188 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.057730 (+438542us) raft_consensus.cc:1685] finished
0709 13:05:42.057755 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.057957 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35790,"raft.queue_time_us":780,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8095,"spinlock_wait_cycles":708992,"thread_start_us":1016520,"threads_started":1,"wal-append.queue_time_us":1168970}
I20260709 13:05:42.015023 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1616) took 2054 ms. Trace:
I20260709 13:05:42.065922 15229 rpcz_store.cc:276] 0709 13:05:39.960259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.536794 (+576535us) service_pool.cc:224] Handling call
0709 13:05:40.537130 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.537535 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:40.537559 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.537972 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.619820 (+1081848us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:41.619874 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.619946 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.014750 (+394804us) raft_consensus.cc:1685] finished
0709 13:05:42.014777 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.014950 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"spinlock_wait_cycles":16708352,"thread_start_us":1081530,"threads_started":1,"wal-append.queue_time_us":1191432}
I20260709 13:05:42.083752 22464 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.076514 22383 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:05:42.085593 22569 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:42.074460 22499 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:42.088299 22499 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:42.088768 22499 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:42.089444 22475 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.006605 22585 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.090848 22585 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:42.072136 22150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.107s	sys 0.194s
W20260709 13:05:42.117034 22150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.107s	sys 0.194s
I20260709 13:05:42.117376 22564 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.091609 22531 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.118546 21942 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.118851 22531 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.119158 21942 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.111904 22394 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.121583 22394 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.121977 21942 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:42.075670 22260 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:42.169210 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.190001 15159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54628 (request call id 1728) took 1644 ms. Trace:
I20260709 13:05:41.259378 15232 rpcz_store.cc:276] 0709 13:05:40.006872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.542298 (+535426us) service_pool.cc:224] Handling call
0709 13:05:40.601144 (+ 58846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.601390 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:40.626836 (+ 25446us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.258434 (+631598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:41.258506 (+    72us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:41.258568 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.258657 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.258798 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26357,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":2472,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6001,"thread_start_us":631172,"threads_started":1}
I20260709 13:05:42.178776 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:42.190431 15159 rpcz_store.cc:276] 0709 13:05:40.545541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.556532 (+ 10991us) service_pool.cc:224] Handling call
0709 13:05:42.189800 (+1633268us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54002baf54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.189943 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439310208}
I20260709 13:05:42.191102 15163 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:42.180169 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1737) took 2495 ms. Trace:
W20260709 13:05:42.180544 15051 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.903s (SENT)
I20260709 13:05:42.192193 15083 rpcz_store.cc:276] 0709 13:05:39.684595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.560743 (+1876148us) service_pool.cc:224] Handling call
0709 13:05:41.676343 (+115600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.676526 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.676553 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:41.676594 (+    41us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:41.676626 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.676679 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.179946 (+503267us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400191c50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.180085 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264776064}
I20260709 13:05:42.176517 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1736) took 2492 ms. Trace:
I20260709 13:05:42.196877 15088 rpcz_store.cc:276] 0709 13:05:39.684205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.160675 (+1476470us) service_pool.cc:224] Handling call
0709 13:05:41.205580 (+ 44905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.694910 (+489330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.176224 (+481314us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b5400118854. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:42.176442 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880496512}
I20260709 13:05:42.196635 22557 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:42.211544 22531 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:42.214349 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1678) took 2260 ms. Trace:
I20260709 13:05:42.214882 15843 rpcz_store.cc:276] 0709 13:05:39.954238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.954397 (+   159us) service_pool.cc:224] Handling call
0709 13:05:39.989873 (+ 35476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.527872 (+537999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.214016 (+1686144us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54004a88d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:42.214274 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236031360}
I20260709 13:05:42.225339 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.227310 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:42.231025 22563 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 1.857s	user 0.015s	sys 0.019s
I20260709 13:05:42.231262 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:42.233875 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2591) took 3797 ms. Trace:
I20260709 13:05:42.234238 15087 rpcz_store.cc:276] 0709 13:05:38.436157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.286606 (+850449us) service_pool.cc:224] Handling call
0709 13:05:39.328321 (+ 41715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.328749 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.328789 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.329396 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.272061 (+942665us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:40.272101 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.272180 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.230689 (+1958509us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b24001762b8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c77d9c4 00007f0d3c77bacb 00007f0d3c936dc9 00007f0d3c936a55 00007f0d3c93609d 00007f0d3f56810e 00007f0d3f57f88c 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221
0709 13:05:42.233583 (+  2894us) raft_consensus.cc:1685] finished
0709 13:05:42.233611 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.233815 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":898,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14987,"spinlock_wait_cycles":113323904,"thread_start_us":869518,"threads_started":1,"wal-append.queue_time_us":877277}
I20260709 13:05:42.236050 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.239532 15844 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:42.240336 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:42.255463 22359 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.006s	sys 0.041s
I20260709 13:05:42.240945 15814 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:42.236608 15087 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:42.247092 22317 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:42.261999 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 546430us
W20260709 13:05:42.257819 22359 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.006s	sys 0.041s
I20260709 13:05:42.254247 22443 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:42.266845 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:42.267099 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:42.270277 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:42.272210 15081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.018s	sys 0.021s
I20260709 13:05:42.272303 15848 raft_consensus.cc:1275] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:05:42.274816 15846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59138 (request call id 1698) took 4271 ms (client timeout 3691 ms). Trace:
W20260709 13:05:42.273639 15081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.019s	sys 0.021s
W20260709 13:05:42.275210 15846 rpcz_store.cc:269] 0709 13:05:38.000451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.000575 (+   124us) service_pool.cc:224] Handling call
0709 13:05:42.271716 (+4271141us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54007c4554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.271861 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4809177856}
I20260709 13:05:42.266664 15814 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:42.277009 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:42.269659 22378 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:42.280560 15155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.004s	sys 0.033s
I20260709 13:05:42.271811 22317 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:42.267377 22499 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.271577 22480 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:42.282119 15155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.004s	sys 0.033s
I20260709 13:05:42.494163 15161 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:42.296681 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2601) took 3373 ms. Trace:
I20260709 13:05:42.535770 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:42.300596 15051 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.569s (SENT)
I20260709 13:05:42.276190 22531 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.343694 22586 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.975s	user 0.007s	sys 0.000s
W20260709 13:05:42.583698 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.607798 15086 rpcz_store.cc:276] 0709 13:05:38.923405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.626680 (+1703275us) service_pool.cc:224] Handling call
0709 13:05:40.632075 (+  5395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.645476 (+ 13401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.296368 (+1650892us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b5400307350. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.296589 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11420928}
I20260709 13:05:42.278309 22359 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:42.413070 22606 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:05:42.531114 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:42.583302 22611 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:05:42.403676 22599 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
W20260709 13:05:42.412348 22592 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:05:42.281145 22443 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:42.429723 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:42.483605 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:42.442466 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:42.601847 15080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.005s	sys 0.014s
I20260709 13:05:42.611270 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3281) took 2336 ms. Trace:
W20260709 13:05:42.575426 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.666568 15080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.015s
I20260709 13:05:42.668676 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1739) took 2789 ms. Trace:
I20260709 13:05:42.669094 15080 rpcz_store.cc:276] 0709 13:05:39.879642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.726811 (+1847169us) service_pool.cc:224] Handling call
0709 13:05:41.727251 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.727870 (+   619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.727915 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.790033 (+ 62118us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.668240 (+878207us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:42.668378 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.668447 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.668485 (+    38us) raft_consensus.cc:1685] finished
0709 13:05:42.668496 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.668613 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66620,"raft.queue_time_us":119,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":60138,"thread_start_us":665654,"threads_started":1,"wal-append.queue_time_us":598335}
I20260709 13:05:42.670836 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.351415 15232 raft_consensus.cc:1240] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:42.568710 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 290786us
W20260709 13:05:42.411782 22604 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.056s	user 0.001s	sys 0.001s
W20260709 13:05:42.410710 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.601948 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.002s	sys 0.015s
W20260709 13:05:42.681622 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.002s	sys 0.017s
I20260709 13:05:42.648217 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.361696 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:42.682229 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.402513 15231 pending_rounds.cc:85] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 11
I20260709 13:05:42.683070 15231 pending_rounds.cc:107] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:05:42.683359 15231 raft_consensus.cc:2889] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:42.353152 22546 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.689579 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.534688 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:42.497474 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:42.550137 22457 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.292918 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:42.570155 22370 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:42.698428 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 256045us
W20260709 13:05:42.705600 22617 log.cc:927] Time spent T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:05:42.707508 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1745) took 2549 ms. Trace:
W20260709 13:05:42.709249 22614 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:05:42.645117 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 344576us
W20260709 13:05:42.710287 22619 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:05:42.710538 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1770) took 2036 ms. Trace:
I20260709 13:05:42.710914 15842 rpcz_store.cc:276] 0709 13:05:40.673811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.677531 (+  3720us) service_pool.cc:224] Handling call
0709 13:05:40.677878 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.692181 (+ 14303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.692210 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.692661 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.239531 (+546870us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:41.239572 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.239650 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.342880 (+1103230us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:42.710261 (+367381us) raft_consensus.cc:1685] finished
0709 13:05:42.710286 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.710482 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13889,"raft.queue_time_us":14585,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5900,"spinlock_wait_cycles":10016512,"thread_start_us":498039,"threads_started":1,"wal-append.queue_time_us":590080}
I20260709 13:05:42.713629 22319 consensus_queue.cc:1059] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:42.601584 15203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.013s	sys 0.008s
I20260709 13:05:42.281824 22472 raft_consensus.cc:2804] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:05:42.353412 22605 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:42.527343 22523 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:42.653925 15154 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:42.531730 22222 consensus_meta.cc:287] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.001s
I20260709 13:05:42.329378 22289 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:42.607334 22480 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:42.283367 22315 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.015s
I20260709 13:05:42.505247 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1721) took 2033 ms. Trace:
I20260709 13:05:42.701370 22353 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:42.503932 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:42.276950 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.735256 15162 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:42.397300 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:42.353621 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:42.682243 15086 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:42.355791 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:42.667001 15083 rpcz_store.cc:276] 0709 13:05:40.274834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.194105 (+1919271us) service_pool.cc:224] Handling call
0709 13:05:42.194563 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:42.195053 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.195094 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:42.195866 (+   772us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.491717 (+295851us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:42.491779 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.491879 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.611037 (+119158us) raft_consensus.cc:1685] finished
0709 13:05:42.611059 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.611215 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1582,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":295329,"threads_started":1,"wal-append.queue_time_us":316974}
I20260709 13:05:42.295953 22378 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:42.738837 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.739456 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.739916 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1733) took 3825 ms. Trace:
I20260709 13:05:42.687721 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.294126 22429 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:42.330672 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1768) took 1687 ms. Trace:
I20260709 13:05:42.746057 15848 rpcz_store.cc:276] 0709 13:05:40.643298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.643439 (+   141us) service_pool.cc:224] Handling call
0709 13:05:42.271963 (+1628524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.274034 (+  2071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.330436 (+ 56402us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54007c4550. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.330605 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3727272832}
I20260709 13:05:42.320071 22564 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:42.691236 15087 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
W20260709 13:05:42.731763 22315 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.007s	sys 0.016s
I20260709 13:05:42.566049 22325 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:42.694362 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 159810us
I20260709 13:05:42.694897 22517 consensus_queue.cc:1048] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:42.696014 22508 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:42.707926 15084 rpcz_store.cc:276] 0709 13:05:40.158114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.028664 (+1870550us) service_pool.cc:224] Handling call
0709 13:05:42.074048 (+ 45384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:42.078049 (+  4001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.078094 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:42.078830 (+   736us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.459883 (+381053us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:42.525977 (+ 66094us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.526782 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.527006 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.707263 (+180257us) raft_consensus.cc:1685] finished
0709 13:05:42.707287 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.707448 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1717,"prepare.queue_time_us":64272,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":4816,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":4723,"thread_start_us":318068,"threads_started":1,"wal-append.queue_time_us":436577}
I20260709 13:05:42.605306 22317 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.717613 15203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.014s	sys 0.008s
W20260709 13:05:42.718724 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.734853 22353 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.663157 15234 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:42.731909 15156 rpcz_store.cc:276] 0709 13:05:40.471401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.503146 (+ 31745us) service_pool.cc:224] Handling call
0709 13:05:40.503439 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.520273 (+ 16834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.520303 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:40.520346 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:40.520375 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.505044 (+1984669us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b540017eb54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:42.505090 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.505198 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38259584}
I20260709 13:05:42.737452 15236 raft_consensus.cc:3060] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:42.724223 22575 raft_consensus.cc:3055] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:42.727377 22565 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:05:42.734329 22527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.007s	sys 0.053s
I20260709 13:05:42.697731 22546 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.731706 22472 raft_consensus.cc:697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:42.732780 22533 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.731549 22480 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:42.757515 15810 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:42.758842 15202 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:42.699116 22457 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.651335 22531 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:42.763454 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 1.03636e+06us
I20260709 13:05:42.773924 15229 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:42.745633 22260 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:42.756723 22325 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:42.735090 22523 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.772830 22575 raft_consensus.cc:740] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:42.776701 22527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.008s	sys 0.053s
I20260709 13:05:42.769767 22317 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:42.760360 22608 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:42.771515 22353 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:42.783752 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83352.6us
I20260709 13:05:42.810838 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:42.788786 22564 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:42.755800 22429 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.796743 15236 raft_consensus.cc:2468] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:42.784967 15813 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:42.775017 22565 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:42.823201 15844 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:42.822017 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.821316 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.828392 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:42.829192 15843 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:42.829386 15839 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:42.816423 22620 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:42.801893 22260 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.834568 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.798832 22533 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:42.809506 22532 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:42.803936 22378 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.831485 22472 consensus_queue.cc:237] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:42.837560 22624 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:42.839552 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.850741 15844 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:42.823112 22629 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:42.810962 22608 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.840572 22630 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:42.838865 22499 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.855451 22389 raft_consensus.cc:2804] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:42.869699 22623 log.cc:927] Time spent T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:05:42.870514 22565 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.874841 22531 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:42.844038 22575 consensus_queue.cc:260] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.875787 22546 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.850852 22532 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:42.877223 22523 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:42.878863 22565 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:42.881273 22608 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.876366 22575 raft_consensus.cc:3060] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:42.882321 22533 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:42.741674 15080 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
W20260709 13:05:42.711953 15051 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.184s (SENT)
W20260709 13:05:42.917403 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96426us
I20260709 13:05:42.918274 15051 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:42.919492 15080 raft_consensus.cc:1240] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:42.750519 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.748903 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
W20260709 13:05:42.344573 15079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56652 (request call id 1742) took 2237 ms (client timeout 1533 ms). Trace:
I20260709 13:05:42.740288 15081 rpcz_store.cc:276] 0709 13:05:38.914879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.565307 (+1650428us) service_pool.cc:224] Handling call
0709 13:05:40.609226 (+ 43919us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:40.632444 (+ 23218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.632682 (+   238us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:40.638962 (+  6280us) log.cc:844] Serialized 52 byte log entry
0709 13:05:41.686800 (+1047838us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b180010aa40. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:42.275681 (+588881us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:42.275721 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.275786 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.735680 (+459894us) raft_consensus.cc:1685] finished
0709 13:05:42.735705 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.739858 (+  4153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19582,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12428,"spinlock_wait_cycles":30886656,"thread_start_us":588686,"threads_started":1,"wal-append.queue_time_us":1848985}
I20260709 13:05:42.735092 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1744) took 2609 ms. Trace:
I20260709 13:05:43.034298 22608 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.766522 15087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:43.035555 15081 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:43.035369 22608 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.036352 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3286) took 2600 ms. Trace:
I20260709 13:05:43.038057 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3287) took 2366 ms. Trace:
I20260709 13:05:42.787005 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.039093 22608 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:43.040966 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 253962us
W20260709 13:05:42.757258 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.786209 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:42.920796 15080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3283) took 2645 ms. Trace:
I20260709 13:05:42.922641 22370 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:43.048638 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:43.049265 22370 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.050405 22569 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:43.051180 22569 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:43.053728 22531 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.054596 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.054780 22531 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.056118 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:42.962324 22260 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.057551 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.057696 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.058161 15844 raft_consensus.cc:1240] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:43.058334 15084 raft_consensus.cc:2376] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20. Re-sending same reply.
I20260709 13:05:43.059150 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.059780 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:42.969969 15127 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.592s (SENT)
I20260709 13:05:42.971385 22457 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:42.977730 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1773) took 2153 ms. Trace:
I20260709 13:05:43.060375 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:42.978773 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1735) took 3295 ms. Trace:
I20260709 13:05:43.061486 15842 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:42.858042 22315 consensus_queue.cc:1059] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.122s
I20260709 13:05:43.066706 22315 raft_consensus.cc:3055] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:43.067116 22315 raft_consensus.cc:740] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.049120 15080 rpcz_store.cc:276] 0709 13:05:40.275351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.918030 (+2642679us) service_pool.cc:224] Handling call
0709 13:05:42.918922 (+   892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.920331 (+  1409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.920642 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:43.060431 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 90782.9us
W20260709 13:05:43.014788 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 302817us
W20260709 13:05:43.047277 15079 rpcz_store.cc:269] 0709 13:05:40.106788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.137930 (+ 31142us) service_pool.cc:224] Handling call
0709 13:05:42.344353 (+2206423us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b540034ab54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:42.344501 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120632064}
I20260709 13:05:43.065647 22546 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.038543 15087 rpcz_store.cc:276] 0709 13:05:40.671212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.036722 (+2365510us) service_pool.cc:224] Handling call
0709 13:05:43.037256 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.037591 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.037613 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.037660 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:43.037701 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.037761 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.037888 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.036748 15081 rpcz_store.cc:276] 0709 13:05:40.436023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.034629 (+2598606us) service_pool.cc:224] Handling call
0709 13:05:43.035158 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.036065 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.036273 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:43.081442 15126 leader_election.cc:336] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.592s (SENT)
I20260709 13:05:43.082505 15086 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:43.084267 15235 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:43.085021 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:43.085502 15228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2769) took 3665 ms (client timeout 1549 ms). Trace:
I20260709 13:05:43.060879 15840 rpcz_store.cc:276] 0709 13:05:40.824532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.032226 (+207694us) service_pool.cc:224] Handling call
0709 13:05:41.032618 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.033056 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.033094 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.033605 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.233395 (+1199790us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b1800480460. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8d9e07 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b 00007f0d3f583d23 00007f0d3f5840d1 00007f0d3f99fa0f
0709 13:05:42.567894 (+334499us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:42.567973 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.568397 (+   424us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.977036 (+408639us) raft_consensus.cc:1685] finished
0709 13:05:42.977076 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.977448 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29331,"raft.queue_time_us":1251,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2532,"spinlock_wait_cycles":38695808,"thread_start_us":334269,"threads_started":1,"wal-append.queue_time_us":1577046}
I20260709 13:05:43.085855 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.053814 22523 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:43.086438 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 248281us
I20260709 13:05:43.086740 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3282) took 2811 ms. Trace:
I20260709 13:05:43.087307 15079 raft_consensus.cc:1275] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:43.087483 22429 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.087855 22260 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:43.087903 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.088855 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.068383 22315 consensus_queue.cc:260] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.064536 22569 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.089604 22389 raft_consensus.cc:697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.072764 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.091028 15162 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:43.034539 15082 rpcz_store.cc:276] 0709 13:05:40.125624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.814956 (+1689332us) service_pool.cc:224] Handling call
0709 13:05:41.815316 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.815562 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:42.278506 (+462944us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.734723 (+456217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:42.734784 (+    61us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:42.734828 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.734909 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.735032 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85190,"prepare.queue_time_us":177926,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":114,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12319,"thread_start_us":829855,"threads_started":2,"wal-append.queue_time_us":510669}
I20260709 13:05:43.061659 15085 rpcz_store.cc:276] 0709 13:05:39.683315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.030822 (+1347507us) service_pool.cc:224] Handling call
0709 13:05:41.118647 (+ 87825us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:41.119011 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:41.378203 (+259192us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.901549 (+523346us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:41.973504 (+ 71955us) log.cc:844] Serialized 52 byte log entry
0709 13:05:41.973639 (+   135us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:41.973671 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.973744 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.978263 (+1004519us) raft_consensus.cc:1685] finished
0709 13:05:42.978307 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.978556 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11059,"prepare.queue_time_us":738940,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":89643,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":5542,"thread_start_us":715285,"threads_started":2,"wal-append.queue_time_us":744771}
I20260709 13:05:43.084762 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1748) took 1406 ms. Trace:
W20260709 13:05:43.096765 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 310505us
W20260709 13:05:43.098841 15228 rpcz_store.cc:269] 0709 13:05:39.420374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.420699 (+   325us) service_pool.cc:224] Handling call
0709 13:05:43.083450 (+3662751us) spinlock_profiling.cc:243] Waited 3.24 s on lock 0x7b54002148d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:43.085225 (+  1775us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7458218368}
I20260709 13:05:43.106415 15844 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:43.107216 15842 raft_consensus.cc:2376] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16. Re-sending same reply.
W20260709 13:05:43.095410 15127 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 1.721s (SENT)
I20260709 13:05:43.109409 15086 rpcz_store.cc:276] 0709 13:05:41.678733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.083556 (+1404823us) service_pool.cc:224] Handling call
0709 13:05:43.084059 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.084418 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.084448 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.084493 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:43.084527 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.084583 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.084705 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.112228 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56652 (request call id 1750) took 1425 ms. Trace:
I20260709 13:05:43.119603 15079 rpcz_store.cc:276] 0709 13:05:41.679308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.086536 (+1407228us) service_pool.cc:224] Handling call
0709 13:05:43.086885 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.102015 (+ 15130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.104348 (+  2333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.122104 15156 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:43.123185 15843 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:43.123790 22315 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.127377 22527 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.128185 22527 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.133638 22469 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:43.134768 22469 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.097247 22471 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:43.090684 22532 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.151481 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.155236 15087 raft_consensus.cc:1240] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:05:43.101418 15088 rpcz_store.cc:276] 0709 13:05:40.275101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.198067 (+1922966us) service_pool.cc:224] Handling call
0709 13:05:42.344741 (+146674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.391011 (+ 46270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.391043 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.391533 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.719258 (+327725us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:42.719318 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.719412 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.980302 (+260890us) raft_consensus.cc:1685] finished
0709 13:05:42.980348 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.084771 (+104423us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540034ab50. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:43.086413 (+  1642us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45782,"raft.queue_time_us":57221,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":6969,"spinlock_wait_cycles":336338688,"thread_start_us":309726,"threads_started":1,"wal-append.queue_time_us":345820}
I20260709 13:05:43.149153 22443 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.167021 15087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3289) took 2490 ms. Trace:
I20260709 13:05:43.167671 15087 rpcz_store.cc:276] 0709 13:05:40.671796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.070851 (+2399055us) service_pool.cc:224] Handling call
0709 13:05:43.118057 (+ 47206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.160175 (+ 42118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.162460 (+  2285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.143440 22457 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:43.176789 15203 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.179006 15128 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.179553 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43970 (request call id 2514) took 3546 ms. Trace:
I20260709 13:05:43.180060 15235 rpcz_store.cc:276] 0709 13:05:39.633162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.633360 (+   198us) service_pool.cc:224] Handling call
0709 13:05:43.083740 (+3450380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.084719 (+   979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.179259 (+ 94540us) spinlock_profiling.cc:243] Waited 3.45 s on lock 0x7b54002148d0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:43.179451 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7934194048}
I20260709 13:05:43.155609 22531 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.183261 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.181315 22608 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:43.190446 15081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3291) took 1628 ms. Trace:
W20260709 13:05:43.190560 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 265910us
I20260709 13:05:43.160755 22523 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.193385 15081 rpcz_store.cc:276] 0709 13:05:41.562301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.098627 (+1536326us) service_pool.cc:224] Handling call
0709 13:05:43.151382 (+ 52755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.190113 (+ 38731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.190144 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.190193 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:43.190224 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.190282 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.190392 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.194554 15083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2606) took 2748 ms. Trace:
I20260709 13:05:43.181830 22429 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:43.177809 22353 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.193289 22608 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.223498 22317 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.221128 15085 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:43.220520 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.228037 15085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2612) took 1021 ms. Trace:
I20260709 13:05:43.228287 15085 rpcz_store.cc:276] 0709 13:05:42.206477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.143772 (+937295us) service_pool.cc:224] Handling call
0709 13:05:43.190119 (+ 46347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.227831 (+ 37712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.227998 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.229372 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.230211 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.231236 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.231618 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.232686 15236 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:05:43.232925 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.233765 15234 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:43.234476 22527 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.234297 22480 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.234529 22325 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.235695 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.236405 15233 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 16.
I20260709 13:05:43.236982 22457 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.237463 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.238484 22353 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.197542 15083 rpcz_store.cc:276] 0709 13:05:40.404515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.922250 (+2517735us) service_pool.cc:224] Handling call
0709 13:05:42.922890 (+   640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.923620 (+   730us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:42.924404 (+   784us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.149314 (+224910us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.149365 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:43.149412 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.149483 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.152380 (+  2897us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6701,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":2959,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"thread_start_us":213938,"threads_started":1,"wal-append.queue_time_us":252405}
I20260709 13:05:43.239672 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.241425 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.243039 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:43.243716 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.249136 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.249909 15840 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:43.259074 15232 raft_consensus.cc:1240] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:05:43.261370 22222 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:43.261641 15086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3294) took 1407 ms. Trace:
I20260709 13:05:43.262153 15086 rpcz_store.cc:276] 0709 13:05:41.853884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.117532 (+1263648us) service_pool.cc:224] Handling call
0709 13:05:43.182515 (+ 64983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.261284 (+ 78769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.261318 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.261382 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:43.261422 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.261476 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.261588 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.263679 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.265399 15846 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:43.265569 22523 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.219269 22429 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.267307 15229 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:43.266353 15081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.273319 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.277139 15083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:43.290906 15082 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:05:43.274152 15086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.227206 22608 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.245148 22480 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.276718 15128 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.271790 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.298105 15080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.297179 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.301308 15085 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 21: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:43.229411 22469 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.302392 15086 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.303862 15082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2614) took 1096 ms. Trace:
I20260709 13:05:43.304183 15082 rpcz_store.cc:276] 0709 13:05:42.207161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.143824 (+936663us) service_pool.cc:224] Handling call
0709 13:05:43.218040 (+ 74216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.299686 (+ 81646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.303819 (+  4133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.302188 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.256736 22457 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.308225 22630 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.314047 22317 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.271201 22517 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
W20260709 13:05:43.306747 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 333536us
I20260709 13:05:43.260478 22370 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.295938 22429 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.274722 22471 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.302747 22325 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.246016 22443 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:43.324767 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 103111us
I20260709 13:05:43.326088 15079 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:43.326300 22443 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.330698 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.330888 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.331558 15840 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.331727 15842 raft_consensus.cc:2410] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:43.332813 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.333393 15847 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:05:43.318156 15083 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:43.334575 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.333982 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.337160 22480 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:43.337278 22640 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:05:43.337742 22480 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.338212 22480 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.315125 22527 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:43.341282 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 69750.3us
I20260709 13:05:43.342140 15080 raft_consensus.cc:2393] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:43.343726 22531 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:43.344188 22531 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.344566 22531 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.340880 22527 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.331648 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.352522 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:43.355104 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.356019 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.356712 15079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2613) took 1149 ms. Trace:
I20260709 13:05:43.357159 15079 rpcz_store.cc:276] 0709 13:05:42.206838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.144768 (+937930us) service_pool.cc:224] Handling call
0709 13:05:43.183592 (+ 38824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.326568 (+142976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.356444 (+ 29876us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540014d554. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:43.356651 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327206656}
W20260709 13:05:43.357800 15126 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.638s (SENT)
W20260709 13:05:43.358915 22289 consensus_meta.cc:287] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.008s
W20260709 13:05:43.359141 15126 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.643s (SENT)
I20260709 13:05:43.360110 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.362031 15841 raft_consensus.cc:1240] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:43.362624 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
W20260709 13:05:43.364007 15126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.364792 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.365798 22630 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:43.366690 22630 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.368811 22499 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:43.369451 22499 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.371531 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.371708 22499 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.372053 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.372689 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.373381 15843 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:43.376137 15083 raft_consensus.cc:2468] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 22.
I20260709 13:05:43.377641 15128 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:43.364637 15126 leader_election.cc:336] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.721s (SENT)
I20260709 13:05:43.381330 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.383131 22644 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.383931 22644 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.348479 15082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115"
I20260709 13:05:43.388775 22289 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:43.389446 22289 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.389637 15086 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:05:43.389951 22289 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.246171 22389 consensus_queue.cc:237] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.391647 22546 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:43.396663 22499 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:43.395346 15084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40386 (request call id 3288) took 2723 ms. Trace:
I20260709 13:05:43.405561 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.408110 15080 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:43.368185 22630 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.411727 15844 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:43.409973 15235 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:43.410488 15840 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
W20260709 13:05:43.420547 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.432403 22499 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.433120 22499 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.434783 15228 raft_consensus.cc:1217] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 15.11->[16.12-18.13]   Dedup: 18.13->[]
I20260709 13:05:43.435351 15228 raft_consensus.cc:1240] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:05:43.436035 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.436311 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.436892 15236 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:43.436961 15233 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:43.437573 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.403854 22471 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.442317 15235 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:05:43.425096 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.427182 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.445086 22520 consensus_queue.cc:1059] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:05:43.421375 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.427609 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:43.439785 22499 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.440320 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.442374 22471 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.443030 15081 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:43.428458 22429 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.445963 22317 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.399892 15084 rpcz_store.cc:276] 0709 13:05:40.671536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.060694 (+2389158us) service_pool.cc:224] Handling call
0709 13:05:43.073653 (+ 12959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.116300 (+ 42647us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:43.151073 (+ 34773us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.394881 (+243808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.394944 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:43.395033 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.395108 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.395261 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29195,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":502,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":144,"thread_start_us":199928,"threads_started":1,"wal-append.queue_time_us":230541}
W20260709 13:05:43.449066 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 148753us
I20260709 13:05:43.430344 22608 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.451740 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.451715 22289 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.453047 22317 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.453470 22644 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.455884 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.456333 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.457038 22527 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.457202 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.458446 15843 raft_consensus.cc:2376] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21. Re-sending same reply.
I20260709 13:05:43.458735 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.459468 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.459993 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.455897 22317 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.461087 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.461488 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.461680 15229 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:43.462175 15235 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:43.462438 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.462759 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.458758 22289 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.464881 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.465539 15233 raft_consensus.cc:2376] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19. Re-sending same reply.
I20260709 13:05:43.466915 22471 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:43.468647 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.469297 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.460749 15161 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:43.472705 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.473563 15156 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.473625 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:43.461380 15232 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:43.478206 22527 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.478833 22527 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.479457 15161 raft_consensus.cc:1240] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:05:43.481047 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:43.481906 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.489178 22527 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.489495 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.489194 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:43.490563 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 63117.9us
I20260709 13:05:43.494207 22480 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.495426 15162 raft_consensus.cc:2393] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:43.497615 22370 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.498622 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.499905 22429 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:43.457126 15127 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641): Timed out: RequestConsensusVote RPC to 127.13.163.196:34641 timed out after 2.140s (SENT)
W20260709 13:05:43.505550 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 48464.6us
I20260709 13:05:43.507164 15237 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
W20260709 13:05:43.511181 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.512890 15232 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:43.516628 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 152007us
W20260709 13:05:43.517849 15126 leader_election.cc:336] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.764s (SENT)
I20260709 13:05:43.518438 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.519578 22429 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:43.519770 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:43.521174 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.525312 22472 raft_consensus.cc:3055] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:43.525820 22472 raft_consensus.cc:740] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.526929 22472 consensus_queue.cc:260] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.527767 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12daa67457954a0cab27e32bf85fa115" is_pre_election: true
I20260709 13:05:43.528334 22472 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:43.528779 15084 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:43.537309 15841 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:43.480831 22499 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.541473 15088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40400 (request call id 2618) took 1333 ms. Trace:
I20260709 13:05:43.538592 22370 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.500883 22480 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.553233 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.554167 15232 raft_consensus.cc:2376] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19. Re-sending same reply.
I20260709 13:05:43.558739 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.559810 22520 consensus_queue.cc:1048] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:43.577041 22370 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:43.577301 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:43.562934 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.561296 22569 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:43.544854 15088 rpcz_store.cc:276] 0709 13:05:42.208373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.183416 (+975043us) service_pool.cc:224] Handling call
0709 13:05:43.227235 (+ 43819us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:43.315664 (+ 88429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.315718 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:43.357148 (+ 41430us) log.cc:844] Serialized 52 byte log entry
0709 13:05:43.447083 (+ 89935us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:43.531291 (+ 84208us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.540997 (+  9706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.541193 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.541244 (+    51us) raft_consensus.cc:1685] finished
0709 13:05:43.541254 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.541398 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22193,"raft.queue_time_us":15088,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2112,"thread_start_us":81799,"threads_started":1,"wal-append.queue_time_us":155993}
I20260709 13:05:43.581862 22635 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.582480 22635 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:43.569453 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 92911.2us
I20260709 13:05:43.562485 22457 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:05:43.578567 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48498.2us
I20260709 13:05:43.449996 22531 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:43.543604 15126 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.913s (SENT)
I20260709 13:05:43.580195 15840 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:43.550962 22630 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:43.606101 15126 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.594589 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.598610 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.597208 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.598193 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:43.594980 15230 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:43.594218 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:43.607596 15841 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:43.609323 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.585280 22508 raft_consensus.cc:2764] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:05:43.595100 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.612288 15232 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:43.612877 15161 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.595729 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:43.585155 22643 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:05:43.614650 22520 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:43.618595 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.624356 15844 raft_consensus.cc:2376] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18. Re-sending same reply.
I20260709 13:05:43.619719 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.622622 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.625509 15843 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.594662 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.622027 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.621897 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.630474 15848 raft_consensus.cc:3060] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.631002 15156 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:43.595254 22471 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:43.594213 22608 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.584775 22635 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.641204 22635 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.615136 22517 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.634074 22644 consensus_queue.cc:1048] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:43.632275 22389 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.635172 15237 raft_consensus.cc:2368] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.633222 15846 raft_consensus.cc:2368] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:43.645378 15814 leader_election.cc:400] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.647974 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.653561 15814 leader_election.cc:403] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:43.648723 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.595130 22317 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:43.654418 15813 leader_election.cc:400] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.584983 22499 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.650906 15158 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:43.657721 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.623950 22520 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:43.652381 15052 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:43.661901 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 48168.1us
I20260709 13:05:43.631758 15155 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
W20260709 13:05:43.663832 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 123069us
I20260709 13:05:43.649317 22480 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:43.655910 22644 consensus_queue.cc:1243] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer 3c28be6f86ea4fb6b79a47d3999f5a82 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:43.675124 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.675993 15844 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:43.665081 22531 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:43.665315 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.665892 22317 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.666435 15847 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:43.667289 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:43.677359 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.667862 22370 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.678030 22289 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:43.655758 22429 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:05:43.668259 22389 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.678977 22222 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:43.680624 15845 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:43.684067 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.669612 22520 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:43.686985 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.673178 22499 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.695094 15229 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:43.696743 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.656518 22471 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.704968 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:43.705637 15844 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.701810 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:43.707084 15163 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.708489 15155 raft_consensus.cc:1275] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:05:43.709421 22517 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:43.709965 22517 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.710878 22517 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.694906 22531 consensus_queue.cc:1059] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:05:43.712853 22531 raft_consensus.cc:3055] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:43.713282 22531 raft_consensus.cc:740] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:43.714286 22531 consensus_queue.cc:260] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:43.715379 15126 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 1.969s (SENT)
I20260709 13:05:43.717087 22480 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:43.717590 22480 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:43.711469 15842 raft_consensus.cc:2410] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
W20260709 13:05:43.695354 15203 connection.cc:256] RPC call timeout handler was delayed by -3.3032107910000832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:05:43.720955 15126 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.089s (SENT)
I20260709 13:05:43.721789 15843 raft_consensus.cc:2468] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:43.686551 22569 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.726218 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.724076 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:43.728188 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.733829 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.736946 15160 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:43.737285 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.737360 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.738651 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.739995 15127 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.741501 15237 raft_consensus.cc:3060] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.742725 22616 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:05:43.743710 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.744267 15229 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:43.743219 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.735597 15158 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:43.756099 15844 raft_consensus.cc:2468] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:43.761359 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.768620 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.769385 15228 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:43.770268 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.774206 22389 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.777513 22289 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:43.777983 22289 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:43.778918 22289 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.780345 22289 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.781628 15233 raft_consensus.cc:2468] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:43.782831 22471 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.790598 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.762507 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.782867 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.787184 15848 raft_consensus.cc:2468] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
W20260709 13:05:43.787487 15126 leader_election.cc:336] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.140s (SENT)
I20260709 13:05:43.792364 15126 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:43.793533 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.796489 15163 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:43.791675 15236 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:43.745241 22443 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.783785 22531 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:43.811419 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 144242us
I20260709 13:05:43.813465 15237 raft_consensus.cc:2468] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:43.814460 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.815649 15236 raft_consensus.cc:2410] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:05:43.819181 22480 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.820676 22480 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:43.826576 22317 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:43.826961 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.829056 15127 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.830812 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:43.833040 22644 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.833838 22644 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:43.836175 22644 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:43.839665 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.783766 22637 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
W20260709 13:05:43.846360 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 204713us
W20260709 13:05:43.850797 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 88342.4us
I20260709 13:05:43.856554 15161 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:43.862370 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:43.862370 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:43.861573 15814 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 13:05:43.873070 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.875401 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:43.882799 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 214922us
W20260709 13:05:43.861501 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 209248us
I20260709 13:05:43.887912 15233 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:43.888769 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 101316us
I20260709 13:05:43.813035 22443 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:43.897704 15236 raft_consensus.cc:2410] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:05:43.850147 22429 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:05:43.900044 15126 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Timed out: RequestConsensusVote RPC to 127.13.163.195:42229 timed out after 2.230s (SENT)
I20260709 13:05:43.900343 15810 leader_election.cc:400] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:43.896889 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.872268 22389 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:43.900506 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:43.907896 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:43.908350 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.908535 15233 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
I20260709 13:05:43.909761 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:43.910449 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:05:43.911201 15235 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:43.911511 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
I20260709 13:05:43.914848 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.920459 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:43.917558 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 134526us
I20260709 13:05:43.888253 22222 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.923728 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:43.924327 15236 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:43.901100 22472 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:43.927453 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.928424 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:43.930281 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.874836 22630 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:05:43.938387 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.916025 22389 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.874850 22523 consensus_queue.cc:1048] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:05:43.945691 15234 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:43.939478 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:43.976974 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.980476 15128 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:43.994369 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:43.947630 22389 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:43.998682 15233 raft_consensus.cc:1240] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:05:43.984994 22472 consensus_queue.cc:1059] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
W20260709 13:05:43.996129 22643 log.cc:927] Time spent T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:05:43.999279 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 50754.5us
I20260709 13:05:43.985582 22429 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:44.002465 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 171744us
W20260709 13:05:44.007200 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.079021 15202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.080276 15128 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.003985 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.099772 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:44.069355 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 148976us
I20260709 13:05:44.100919 15232 raft_consensus.cc:2376] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20. Re-sending same reply.
I20260709 13:05:44.008821 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:44.101069 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 161678us
I20260709 13:05:44.018770 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:44.099879 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 238399us
W20260709 13:05:44.021356 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 121357us
I20260709 13:05:43.987680 22630 raft_consensus.cc:2764] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:05:43.981694 22222 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:43.987301 22644 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:44.102128 15230 raft_consensus.cc:2376] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20. Re-sending same reply.
I20260709 13:05:44.017396 22359 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.017728 22667 raft_consensus.cc:2804] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:44.008047 22523 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:44.022715 22378 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:44.021564 22629 raft_consensus.cc:2804] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:44.008435 22649 consensus_queue.cc:1059] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:05:44.008225 22671 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:44.120285 22569 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:44.120172 22644 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.121344 22677 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:44.121708 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.122895 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.163254 15233 raft_consensus.cc:2368] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:44.123207 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.124500 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.150906 22673 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:44.125952 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.020318 22443 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:44.170491 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 161814us
I20260709 13:05:44.129469 15237 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:44.138767 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.008280 22675 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:44.008085 22672 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:44.146747 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.013183 22669 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:44.150051 15236 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:44.152989 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:44.155409 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.150817 22378 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.165086 15235 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:44.125918 22317 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.166930 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.162763 22667 raft_consensus.cc:697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.166710 22677 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.174188 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.152067 22523 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:44.162490 22649 raft_consensus.cc:3055] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:44.162940 22629 raft_consensus.cc:697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:44.179980 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.188735 15234 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:44.183373 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.169912 22673 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.184916 15232 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 15.
W20260709 13:05:44.190318 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 243459us
I20260709 13:05:44.190834 15051 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.167414 22429 consensus_queue.cc:1059] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:05:44.182001 22378 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:44.196661 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 50597.2us
I20260709 13:05:44.186715 22569 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:44.186895 22649 raft_consensus.cc:740] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:44.154244 22359 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.188894 22675 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:44.199502 15201 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:44.193554 22673 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:44.186309 22317 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.224469 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.208736 22443 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:44.232465 22663 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.393s	user 0.006s	sys 0.000s
W20260709 13:05:44.253357 15129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.259106 22527 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:44.262385 22677 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.265753 15839 raft_consensus.cc:1240] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:44.266917 22359 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.268608 22317 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.269642 22644 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.269966 22378 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.273288 22649 consensus_queue.cc:260] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.274742 15848 pending_rounds.cc:85] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting all ops after (but not including) 11
I20260709 13:05:44.276592 15230 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:44.278941 15232 raft_consensus.cc:3055] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:44.278764 22565 consensus_queue.cc:1059] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:44.226876 22666 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:05:44.279711 22523 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.281893 22667 consensus_queue.cc:237] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:44.283272 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 81360us
I20260709 13:05:44.219563 22429 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:44.321760 15848 pending_rounds.cc:107] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:05:44.291218 22353 raft_consensus.cc:3055] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:44.293334 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.294060 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.294772 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.331974 15848 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:44.253911 22629 consensus_queue.cc:237] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.231470 22675 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.306375 15229 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 16.
I20260709 13:05:44.317168 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.339411 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.320717 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.340318 15233 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:44.333864 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.322481 15232 raft_consensus.cc:740] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:44.326268 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.302524 22527 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.220355 22520 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.328370 15163 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:44.334528 15846 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:44.339727 15234 raft_consensus.cc:2393] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 19: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:44.337029 15842 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:44.346189 22678 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:05:44.350025 22678 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:44.350641 22678 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.351702 22678 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.353860 22662 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:44.354472 22662 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.357023 22662 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.357411 22649 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:44.358752 22673 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.334086 22353 raft_consensus.cc:740] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:44.337827 22565 raft_consensus.cc:3055] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:44.364923 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.364799 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.365970 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.377876 15236 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:44.373457 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.362917 22523 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:44.376578 15841 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:44.334431 22480 consensus_queue.cc:1059] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:05:44.380405 15233 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:44.377928 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.345674 22527 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:44.362680 22457 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.384545 15230 raft_consensus.cc:2468] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18.
I20260709 13:05:44.383803 15234 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:44.377046 22565 raft_consensus.cc:740] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:44.339826 22429 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.375056 15232 consensus_queue.cc:260] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.363888 22662 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:44.394611 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.395726 22569 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:44.395524 22644 consensus_queue.cc:1059] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:44.375399 22531 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.385753 22289 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.005s
W20260709 13:05:44.405596 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 227557us
W20260709 13:05:44.405810 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84825.6us
I20260709 13:05:44.409924 15232 raft_consensus.cc:3060] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:44.417047 15236 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
W20260709 13:05:44.420373 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.410003 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 214256us
I20260709 13:05:44.423290 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:44.424139 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 43997.5us
W20260709 13:05:44.425308 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.430166 15847 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:44.425940 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:44.430676 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:44.440990 22317 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:44.445039 22575 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:44.445555 22575 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:44.445953 22575 raft_consensus.cc:3060] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:44.435427 22687 raft_consensus.cc:3055] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:44.447279 22687 raft_consensus.cc:740] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:44.448254 22687 consensus_queue.cc:260] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.449430 22378 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.449615 22687 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:44.433017 22353 consensus_queue.cc:260] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.452157 22353 raft_consensus.cc:3060] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:44.423136 22531 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.454246 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.455236 22359 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:44.455657 22531 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.456903 22359 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:44.458755 22569 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:44.460153 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:44.419955 22673 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.463148 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:44.469071 22359 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:44.473637 22359 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:44.474102 22359 raft_consensus.cc:3060] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:44.477270 22565 consensus_queue.cc:260] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:44.472962 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47140.4us
W20260709 13:05:44.486075 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 66287.2us
I20260709 13:05:44.388749 22457 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.500707 15232 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:05:44.497202 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:44.501499 15232 pending_rounds.cc:85] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 14
I20260709 13:05:44.502039 15232 pending_rounds.cc:107] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:05:44.502450 15232 raft_consensus.cc:2889] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:44.504886 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.507428 15127 leader_election.cc:400] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:44.507899 15127 leader_election.cc:403] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:44.513826 15127 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.511118 22674 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:44.528507 22289 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:44.507050 22673 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:05:44.510567 22389 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.542639 22685 raft_consensus.cc:2804] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:44.541455 22674 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:44.536283 22688 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:44.552526 22565 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:44.554272 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.554327 15160 raft_consensus.cc:1240] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 LEADER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:44.556963 15127 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.559917 15842 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:44.560039 22289 raft_consensus.cc:2804] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:44.560518 22289 raft_consensus.cc:697] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:44.561892 15127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.561424 22289 consensus_queue.cc:237] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.564605 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.564862 15841 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:44.565326 22389 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.565357 15163 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:44.565359 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.567164 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.567837 15842 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 16: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:44.569527 22429 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.557859 22674 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:44.575822 22681 raft_consensus.cc:3055] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:44.576380 22681 raft_consensus.cc:740] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.577497 22681 consensus_queue.cc:260] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.578722 22457 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.551062 22520 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:44.579174 22681 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:44.572440 22499 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:44.582790 22499 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.583967 22499 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.585631 22499 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:44.587419 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.588403 22457 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:44.588879 22457 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.589301 22457 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:44.558494 22688 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.594015 22688 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:44.595598 22668 log.cc:927] Time spent T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.599s	user 0.000s	sys 0.002s
I20260709 13:05:44.598080 22457 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.601095 22685 raft_consensus.cc:697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.601408 22457 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.601739 15162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2802) took 1005 ms. Trace:
I20260709 13:05:44.602393 22685 consensus_queue.cc:237] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.588472 15054 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:44.614152 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 353193us
I20260709 13:05:44.621156 22671 raft_consensus.cc:3055] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:44.621675 22671 raft_consensus.cc:740] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:44.622650 22671 consensus_queue.cc:260] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.623834 22671 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:44.624048 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.624727 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.626438 22575 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.628383 15236 raft_consensus.cc:1275] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:44.602233 15162 rpcz_store.cc:276] 0709 13:05:43.595817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.608939 (+ 13122us) service_pool.cc:224] Handling call
0709 13:05:43.631064 (+ 22125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.906819 (+275755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.906864 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.932396 (+ 25532us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.187732 (+255336us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:44.187773 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.601287 (+413514us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b54001c61d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:44.601345 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.601401 (+    56us) raft_consensus.cc:1685] finished
0709 13:05:44.601416 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.601665 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32284,"raft.queue_time_us":83960,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":14782,"spinlock_wait_cycles":145534208,"thread_start_us":357776,"threads_started":2,"wal-append.queue_time_us":335738}
I20260709 13:05:44.630339 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.635138 22527 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.636339 15237 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:44.639377 22359 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.640262 15126 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.639829 22575 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.641383 22480 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:44.642203 22527 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.643925 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.644778 22480 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:44.645053 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:44.644778 22359 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.648428 15126 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:44.649873 22575 raft_consensus.cc:2804] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:44.650899 22575 raft_consensus.cc:697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:44.656183 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67785.9us
I20260709 13:05:44.664104 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.666262 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.667033 15158 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 25.
I20260709 13:05:44.671862 22565 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:44.672484 22565 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.679472 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.689378 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:44.613657 22317 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:44.700383 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 237322us
W20260709 13:05:44.706117 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 49056us
I20260709 13:05:44.707376 15054 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.707680 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.711046 15845 raft_consensus.cc:1240] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 13:05:44.697796 22359 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:44.716724 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:44.701939 22565 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:44.735020 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.735868 15162 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:44.737536 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:44.652091 22575 consensus_queue.cc:237] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.741438 15158 raft_consensus.cc:1275] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:44.742426 22569 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:44.743629 22639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.008s	sys 0.013s
W20260709 13:05:44.744024 22639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.008s	sys 0.013s
I20260709 13:05:44.744972 22662 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:05:44.739468 22317 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:44.743647 22569 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:44.747251 22569 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:44.749476 22696 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:44.751564 22674 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.728569 22359 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.753073 22359 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:44.757033 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 108695us
I20260709 13:05:44.728494 22649 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:44.766651 22649 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:44.767616 22649 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:44.768183 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 51256.9us
I20260709 13:05:44.757121 22569 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:44.777155 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.777606 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.758564 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.779623 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:44.780325 15126 leader_election.cc:400] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:44.781322 15126 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.782646 22673 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:44.783330 22673 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:44.783927 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:44.784789 22443 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:44.785810 22569 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.786801 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:44.797511 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.802675 22688 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.805380 22688 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:44.763864 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 56287.1us
I20260709 13:05:44.763736 22699 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:44.762787 15847 raft_consensus.cc:2468] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:44.786816 15126 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:44.818823 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 97098.7us
I20260709 13:05:44.821214 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:44.822053 15158 raft_consensus.cc:3060] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:44.823285 15052 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:44.818701 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 344974us
I20260709 13:05:44.733166 22644 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:44.844624 15846 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:44.742259 22678 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:05:44.845743 22289 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:44.848397 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.848421 22673 consensus_queue.cc:1059] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:44.847131 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 60373.2us
I20260709 13:05:44.828717 22569 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:05:44.857730 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 360034us
W20260709 13:05:44.859227 15203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.859464 15201 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:44.863162 22688 raft_consensus.cc:2804] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:44.863662 22688 raft_consensus.cc:697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:44.864598 22688 consensus_queue.cc:237] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.865943 22687 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:44.826562 22677 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:44.872532 22677 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:44.896116 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 76300.5us
I20260709 13:05:44.856921 22569 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:44.905722 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.908224 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.896601 22644 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:44.856443 22359 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:44.909643 15159 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
W20260709 13:05:44.923465 15813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.911651 15811 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.911543 15127 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.917938 15814 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:44.949287 15846 raft_consensus.cc:1275] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:05:44.949442 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126306us
W20260709 13:05:44.943027 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 95468.9us
I20260709 13:05:44.942874 15158 raft_consensus.cc:2468] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:44.957408 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:44.960978 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:44.926682 22359 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.945261 22677 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.938861 22649 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:44.909430 22569 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:44.919483 22644 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:44.916747 22695 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:05:44.991263 15814 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:44.995774 22667 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:45.010991 22667 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:45.011475 22667 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:45.013608 22678 raft_consensus.cc:3055] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:45.014045 22678 raft_consensus.cc:740] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:45.014757 22508 raft_consensus.cc:2804] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:45.015350 22508 raft_consensus.cc:697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:45.014958 22678 consensus_queue.cc:260] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.016466 22678 raft_consensus.cc:3060] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:44.997713 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.017354 22689 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:45.019747 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.020859 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.021591 15158 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.022675 15161 raft_consensus.cc:1240] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:44.961939 22517 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:44.946219 22674 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:44.914134 22523 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:05:45.032670 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.033553 15847 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:45.033654 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.917946 22532 consensus_queue.cc:1048] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:05:44.953892 22671 raft_consensus.cc:2804] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:44.962584 15051 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:44.989629 22575 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:45.008805 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:44.936527 22685 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:45.024603 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:44.959440 22289 consensus_queue.cc:1048] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 13:05:45.037159 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.040426 15162 raft_consensus.cc:1240] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:45.045130 15847 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:45.054606 15158 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:44.998927 22662 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:45.016294 22508 consensus_queue.cc:237] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.060719 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:45.052264 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.061244 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:45.025933 22517 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.068403 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.051669 22690 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.086738 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.072124 22517 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:45.093953 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.088989 22690 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.080065 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:45.081076 15161 raft_consensus.cc:1240] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:45.081228 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 132105us
I20260709 13:05:45.081926 22708 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:45.027158 22520 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:05:45.049700 22629 consensus_queue.cc:1048] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:05:45.085441 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.081169 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:45.045107 22575 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.098383 15842 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:45.075911 22689 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:45.107609 15229 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:45.138103 15203 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.137130 15230 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:45.139371 15846 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:45.139770 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.140497 15236 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:45.140733 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.142426 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.145007 22289 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:45.146467 22690 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.143282 15161 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 25.
I20260709 13:05:45.148241 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.153175 22705 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.153347 22527 consensus_queue.cc:1059] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:45.154757 22644 raft_consensus.cc:3055] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:45.155211 22644 raft_consensus.cc:740] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:45.136153 22708 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:45.156039 22644 consensus_queue.cc:260] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.157157 22708 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.118104 22698 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:45.158489 22698 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.159505 15842 raft_consensus.cc:1275] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:45.139187 22575 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:45.091566 15155 raft_consensus.cc:2393] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.106954 15156 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.104975 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.067384 22714 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:45.073915 15813 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:45.163942 15234 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 20. Ops: [18.13-18.13]
W20260709 13:05:45.109514 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 64908.5us
I20260709 13:05:45.165802 15052 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:45.103101 22569 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.112178 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.173666 22569 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 26 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:45.112876 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 136840us
I20260709 13:05:45.173923 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.174768 15234 raft_consensus.cc:3060] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:45.113826 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.175848 15235 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:45.070725 22671 raft_consensus.cc:697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:45.132969 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.177294 22671 consensus_queue.cc:237] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.171279 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.178637 15155 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:45.171795 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.181375 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.174458 22667 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.177773 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.186937 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:45.182021 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 222031us
I20260709 13:05:45.182610 15236 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:45.184357 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.165699 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.124621 15840 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
W20260709 13:05:45.168778 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.189990 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.193483 15156 raft_consensus.cc:2410] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:05:45.193809 15161 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:45.201273 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 68425.7us
I20260709 13:05:45.186769 22715 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:45.210558 15051 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.214394 22359 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:45.214522 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.219425 15156 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:45.215427 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.185750 22698 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:45.215629 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 143250us
I20260709 13:05:45.203187 22569 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:45.202510 22699 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:45.228549 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.228708 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.238224 15156 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:45.238279 15161 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:45.185091 22480 consensus_queue.cc:1048] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:45.240952 15162 raft_consensus.cc:1240] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:45.244304 15162 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:05:45.244722 15162 pending_rounds.cc:85] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 16
I20260709 13:05:45.245359 15162 pending_rounds.cc:107] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 13:05:45.245680 15162 raft_consensus.cc:2889] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.247383 15156 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 26.
W20260709 13:05:45.258800 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.230386 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.219363 22715 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:45.280069 15156 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:45.187775 22714 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:45.265592 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:45.193167 22662 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:05:45.225147 22520 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:45.286631 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.299949 15159 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.229300 22532 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:45.203886 22565 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.236876 22705 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:45.289698 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.265259 22649 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.301415 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:45.321543 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58935.8us
I20260709 13:05:45.321717 15126 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.288765 22700 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.287187 22715 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:45.290082 22697 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.322657 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.326793 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57290.3us
I20260709 13:05:45.326257 15232 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:45.264147 22699 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:45.346737 15128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.348008 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:45.350246 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 235125us
I20260709 13:05:45.363406 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:45.356428 15234 raft_consensus.cc:2468] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:45.326380 22629 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:45.383819 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.387308 15233 raft_consensus.cc:1275] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:45.388325 15160 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:45.325877 22443 consensus_queue.cc:1059] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:05:45.350962 22688 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:45.312104 22532 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.390095 15159 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:45.354177 22480 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.321542 22698 consensus_queue.cc:1059] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:45.361739 22678 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.397892 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.323226 22705 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.389762 22644 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:45.412511 15155 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:05:45.395020 22289 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:45.398084 22688 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:45.423058 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 195420us
I20260709 13:05:45.405943 22527 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.349198 22700 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.424656 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:45.424515 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 237662us
I20260709 13:05:45.407282 22443 consensus_queue.cc:1243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 3d45101e0851484690b4f03a4dcdb63d log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:05:45.428947 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45751.7us
W20260709 13:05:45.429015 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65717.2us
I20260709 13:05:45.433168 22671 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:45.418690 22667 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.354918 22697 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.417387 22689 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.421746 22649 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:45.438572 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 59481.1us
W20260709 13:05:45.440315 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.446470 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.447763 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 146440us
I20260709 13:05:45.411782 22480 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.428306 22688 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:45.444059 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.430322 22698 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.437512 22457 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.451906 15842 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:45.462548 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.470141 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:45.471426 15203 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.474831 15845 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:45.428695 22680 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:05:45.434821 22527 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.460943 15201 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.451097 22729 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:45.484010 22687 raft_consensus.cc:3055] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:45.440099 22689 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.518610 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:45.525691 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54364.7us
I20260709 13:05:45.526625 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:45.482046 22698 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.514532 22729 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:45.545876 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 85469.1us
I20260709 13:05:45.546921 15201 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.496605 22565 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.520239 22687 raft_consensus.cc:740] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:45.515615 22714 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:45.567361 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.528690 22699 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.573915 22705 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.489396 22457 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.574728 22730 consensus_queue.cc:1059] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:45.576596 22517 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.577981 22480 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.574009 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:45.600903 22703 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.726s	user 0.005s	sys 0.000s
I20260709 13:05:45.601765 22708 raft_consensus.cc:2804] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:45.602289 22708 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:45.602700 22708 raft_consensus.cc:3060] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:45.603538 22719 log.cc:927] Time spent T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:05:45.604423 15847 raft_consensus.cc:1240] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Rejecting Update request from peer 4b633b84351f40de9437e3d532fe6fa7 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:05:45.582667 22697 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.549966 22729 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:45.586184 22714 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.588233 22565 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.602696 22508 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:45.620452 15157 pending_rounds.cc:85] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 12
I20260709 13:05:45.605123 22705 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:45.621760 15157 pending_rounds.cc:107] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:05:45.622059 15157 raft_consensus.cc:2889] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.583305 22575 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:45.631358 15200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.624986 22637 consensus_queue.cc:1048] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:05:45.646365 22565 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:45.646754 22520 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.647264 22520 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.647809 22517 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:45.655877 15052 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:45.676434 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 149905us
I20260709 13:05:45.621574 22697 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:45.625217 22508 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:45.627462 22532 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:45.682791 15158 raft_consensus.cc:2393] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.625497 22705 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.689255 15159 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:45.689409 22734 consensus_queue.cc:1059] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:45.641396 22649 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:45.698122 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.675618 22674 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:45.621191 22714 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.711201 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.712944 15845 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:45.717144 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.717437 22527 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:45.718776 22735 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:05:45.719930 22748 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:45.720739 22747 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:45.725390 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 178477us
I20260709 13:05:45.690749 22508 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:45.702953 22705 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:45.731491 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.732196 15201 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:45.734866 15839 raft_consensus.cc:1275] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:45.732342 15842 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:45.736341 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:45.705518 22700 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.736662 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.738935 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:45.739925 15844 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:45.677187 22687 consensus_queue.cc:260] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:45.749801 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67597.7us
I20260709 13:05:45.736536 22674 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:45.742316 22742 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:45.757253 22752 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:45.758737 22675 raft_consensus.cc:3055] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:45.742789 22527 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:45.735790 22747 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.756461 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:45.762828 22733 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:45.763108 15843 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:45.769480 22674 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.806454 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:45.810938 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 54622.5us
I20260709 13:05:45.811383 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.742714 22751 consensus_queue.cc:1059] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 13:05:45.814088 15228 raft_consensus.cc:1240] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:45.815838 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.777709 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.817086 15231 raft_consensus.cc:2393] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:45.770605 22698 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.777326 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.823088 15233 raft_consensus.cc:2376] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 18. Re-sending same reply.
I20260709 13:05:45.824373 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.824826 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.825735 15232 raft_consensus.cc:2393] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:45.826099 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.862193 15233 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:45.866868 15128 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:45.770231 22675 raft_consensus.cc:740] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:45.814249 22457 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.829813 22750 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.842924 22745 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:45.824455 22757 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:45.871191 22757 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:45.871642 22757 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:45.845712 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.846818 22714 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:45.847779 22707 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.818s	user 0.004s	sys 0.006s
I20260709 13:05:45.848196 15161 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:45.809087 22649 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.849448 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.851732 15126 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:45.890322 15231 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:45.890316 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.852238 22687 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:45.853384 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.860131 22708 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.823882 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:45.903877 15232 raft_consensus.cc:2393] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
W20260709 13:05:45.827936 22569 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.592s	user 0.014s	sys 0.001s
I20260709 13:05:45.870018 22750 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.867584 15128 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:45.886751 15228 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:45.911650 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.887591 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.888810 15843 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:45.912667 15158 raft_consensus.cc:2393] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:45.892799 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.913581 15160 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 22.
I20260709 13:05:45.895639 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.898135 15848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1794) took 1624 ms. Trace:
I20260709 13:05:45.898737 22289 raft_consensus.cc:2804] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:45.902495 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.914932 15848 rpcz_store.cc:276] 0709 13:05:44.273993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.274132 (+   139us) service_pool.cc:224] Handling call
0709 13:05:44.274489 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.663248 (+388759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.663279 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.663888 (+   609us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.876861 (+212973us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:44.876934 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.877008 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.828940 (+951932us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:45.897800 (+ 68860us) raft_consensus.cc:1685] finished
0709 13:05:45.897824 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.898073 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9260,"raft.queue_time_us":15524,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3006,"spinlock_wait_cycles":29838080,"thread_start_us":514169,"threads_started":2,"wal-append.queue_time_us":296214}
I20260709 13:05:45.914301 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:45.920810 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53984.2us
I20260709 13:05:45.924495 15232 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:45.903025 22757 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.927021 22499 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:45.928975 22757 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.929105 22747 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.819072 22733 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.931250 22733 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.933422 22289 raft_consensus.cc:697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:45.902712 22729 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.934629 22289 consensus_queue.cc:237] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.936542 22729 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.888453 22705 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.939132 22705 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.942435 22750 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.945147 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:45.950430 22708 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:45.950703 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:45.953312 22715 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.005s
I20260709 13:05:45.953755 22644 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:45.954258 22644 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.953897 22715 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.954679 22644 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:45.960084 22508 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.916960 22569 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.917400 22575 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:45.963542 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:45.971697 15159 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:45.964655 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:45.975718 15160 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:45.975940 22644 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.977116 15160 pending_rounds.cc:85] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 12
I20260709 13:05:45.977438 15160 pending_rounds.cc:107] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:45.977684 15160 raft_consensus.cc:2889] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.979305 22644 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.909693 22688 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:45.982033 15229 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 26: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:45.984920 22520 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:45.984977 22644 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.985689 22644 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.986585 15232 raft_consensus.cc:1275] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:45.989353 22644 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:45.965812 22472 raft_consensus.cc:3055] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:45.966688 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:45.969815 22675 consensus_queue.cc:260] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:45.991323 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.007704 15231 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:45.991739 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.011492 15054 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:46.013827 15201 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.015910 15159 raft_consensus.cc:2468] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:46.019186 22671 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.019471 22565 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:46.019703 22671 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.020332 22527 consensus_queue.cc:1059] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:46.021374 22565 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:46.021394 22527 raft_consensus.cc:3055] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:46.022009 22527 raft_consensus.cc:740] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:46.022825 22671 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.021951 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.023145 22527 consensus_queue.cc:260] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.024569 22755 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:46.023175 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:46.025033 15813 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.025418 15847 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:46.024065 22741 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.026821 22741 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.028506 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.024209 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.025525 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.021912 22565 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.030494 22520 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:46.030539 22565 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:46.031474 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.033444 22741 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:46.034066 15813 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.035370 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.029209 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.035830 15813 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:46.039976 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.040103 22520 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:46.040769 15813 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.041317 15813 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.034839 22527 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:46.040937 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:46.042714 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49654.9us
I20260709 13:05:46.044296 15203 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.045192 22752 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:46.045814 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.045742 22752 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.046294 22752 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:46.047605 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.048297 15844 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:46.049787 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.006733 22472 raft_consensus.cc:740] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:46.013074 22644 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:46.015043 22705 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:46.052518 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:46.047870 22708 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:05:46.061069 22708 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 26 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:46.062273 22708 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.064916 22675 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:46.017167 22750 consensus_queue.cc:1048] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:05:46.059775 22644 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.070915 22644 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:46.016793 22569 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:05:46.015365 22688 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:46.054239 22748 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:46.054463 22700 raft_consensus.cc:2804] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:46.014168 22575 consensus_queue.cc:1048] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:05:46.051524 22520 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:46.095497 22732 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.638s	user 0.000s	sys 0.005s
I20260709 13:05:46.095651 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:46.096722 15162 raft_consensus.cc:2393] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 26: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:46.097955 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.099323 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.100142 15162 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:46.078150 22705 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.081848 22317 raft_consensus.cc:3055] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:46.110025 22317 raft_consensus.cc:740] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:46.110973 22317 consensus_queue.cc:260] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.111990 22317 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:46.112975 22705 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:46.091174 22715 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.101826 22700 raft_consensus.cc:697] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:46.104022 15201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:46.121279 22737 log.cc:927] Time spent T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.657s	user 0.003s	sys 0.000s
I20260709 13:05:46.121616 22700 consensus_queue.cc:237] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:46.123194 22718 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.701s	user 0.003s	sys 0.000s
W20260709 13:05:46.124738 22739 log.cc:927] Time spent T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.599s	user 0.001s	sys 0.001s
I20260709 13:05:46.088306 22748 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.126395 22748 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:46.126819 22723 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.702s	user 0.004s	sys 0.000s
W20260709 13:05:46.130970 15128 leader_election.cc:336] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.131610 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.133832 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:46.134799 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:46.135200 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:46.136699 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:46.137442 15237 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:46.137676 22743 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.564s	user 0.002s	sys 0.000s
I20260709 13:05:46.138478 22671 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:46.142482 22722 log.cc:927] Time spent T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.755s	user 0.002s	sys 0.000s
W20260709 13:05:46.144048 22727 log.cc:927] Time spent T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.699s	user 0.001s	sys 0.000s
I20260709 13:05:46.145432 14983 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 18, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:46.146884 22731 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.605s	user 0.001s	sys 0.001s
I20260709 13:05:46.147341 22472 consensus_queue.cc:260] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.147282 14983 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 21, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 21 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.148463 22472 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:46.151099 14983 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.153352 22662 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:46.153560 15128 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.153931 22662 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.102833 15161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3226) took 1026 ms. Trace:
I20260709 13:05:46.157701 15161 rpcz_store.cc:276] 0709 13:05:45.076154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.239427 (+163273us) service_pool.cc:224] Handling call
0709 13:05:45.847834 (+608407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.889485 (+ 41651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.102545 (+213060us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54001c6bd0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e38ad 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:46.102758 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398378880}
I20260709 13:05:46.134747 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:46.159649 22508 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.160221 22508 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.167063 22742 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:46.167682 22742 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:46.170495 22689 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.174055 22662 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:46.177136 15128 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.177855 15128 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.178761 22689 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:46.180150 15128 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.160665 22508 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:46.183959 22725 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.741s	user 0.000s	sys 0.003s
I20260709 13:05:46.185436 22508 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
I20260709 13:05:46.189486 22359 raft_consensus.cc:3055] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:46.189883 22359 raft_consensus.cc:740] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:46.190915 22359 consensus_queue.cc:260] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.192233 22359 raft_consensus.cc:3060] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:46.194013 15163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57510 (request call id 3220) took 1174 ms. Trace:
I20260709 13:05:46.194404 15163 rpcz_store.cc:276] 0709 13:05:45.019277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.023510 (+  4233us) service_pool.cc:224] Handling call
0709 13:05:45.023842 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.029073 (+  5231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.029105 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.063821 (+ 34716us) log.cc:844] Serialized 52 byte log entry
0709 13:05:45.353701 (+289880us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:45.353746 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.353813 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.183270 (+829457us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3f57f86e 00007f0d3f57ebb6 00007f0d3f57e7d8 00007f0d3f588221 00007f0d3f5881d9 00007f0d3f588169
0709 13:05:46.193671 (+ 10401us) raft_consensus.cc:1685] finished
0709 13:05:46.193690 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.193941 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1410,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2891,"spinlock_wait_cycles":322235136,"thread_start_us":247905,"threads_started":1,"wal-append.queue_time_us":306052}
I20260709 13:05:46.197803 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:46.183156 15128 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.202404 15155 raft_consensus.cc:1240] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:46.203210 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:46.203536 15202 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.204607 15202 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.205188 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:46.206703 15202 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.207268 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.207677 22741 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.208218 22741 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.208997 22697 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:46.210317 22741 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:46.210608 15202 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.211319 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.211596 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:46.211875 15202 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.212211 15847 raft_consensus.cc:2410] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:05:46.212464 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:46.212857 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.214016 15237 raft_consensus.cc:1275] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:46.221727 15842 raft_consensus.cc:1275] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:05:46.225749 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.220301 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.224498 15051 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.221005 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:46.213497 15202 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.203536 15128 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.088989 15159 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:46.233881 15847 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
W20260709 13:05:46.225369 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:46.252723 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49215.5us
I20260709 13:05:46.152570 14983 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.220201 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:46.256996 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.235212 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:46.233453 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.260509 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:46.260823 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 47342.7us
W20260709 13:05:46.261861 15202 leader_election.cc:336] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.262410 15202 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.265642 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.266486 15848 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:46.266968 15840 raft_consensus.cc:1275] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:46.255077 14983 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 19, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 19 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:46.261402 15237 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
W20260709 13:05:46.253850 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.270514 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.270970 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.272130 15159 raft_consensus.cc:2468] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 23.
I20260709 13:05:46.275986 15203 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:46.280138 15202 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.280597 15157 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:05:46.280623 15202 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.284176 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.291184 15201 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.289964 14983 catalog_manager.cc:5697] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.298993 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:46.281072 22741 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:46.213403 22748 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:46.308504 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 84075.1us
I20260709 13:05:46.283020 22688 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:46.301273 14983 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 19, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 19 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.302999 22747 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:46.313297 22747 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.214957 22532 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:05:46.291744 22687 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:46.292928 22757 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.317889 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.278774 22685 consensus_queue.cc:1048] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:05:46.303298 22457 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:46.317788 22697 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:46.296232 22705 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.321764 22741 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:46.329596 22755 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:46.320989 22747 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.364920 22752 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.366379 15229 raft_consensus.cc:2368] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:46.368006 22457 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.306082 22714 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:05:46.343590 22733 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:46.382103 22705 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.382328 15810 leader_election.cc:400] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:46.340343 22680 consensus_queue.cc:1048] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:05:46.330149 22688 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:46.358156 22685 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:46.361675 15235 raft_consensus.cc:1240] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:05:46.363000 22689 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:46.372466 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.390679 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.311679 22742 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.383780 22688 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.382736 15810 leader_election.cc:403] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:46.384704 15232 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:46.333431 22757 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.364310 22644 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.339125 22629 consensus_queue.cc:1059] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:05:46.413007 22629 raft_consensus.cc:3055] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:46.413427 22629 raft_consensus.cc:740] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:46.414294 22629 consensus_queue.cc:260] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.415616 22629 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:46.408444 22757 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.419436 15233 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:46.425093 15051 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.350914 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:46.344985 22750 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:46.413094 22644 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.443807 22317 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:46.368625 22457 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.392828 22733 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:46.417922 22747 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.388823 22685 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.411958 22748 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.441859 22680 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:05:46.450290 22359 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.463325 22359 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.463915 22662 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:46.465628 22359 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.449739 22569 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:46.469233 22569 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:46.481850 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 201717us
W20260709 13:05:46.482947 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.486191 15230 raft_consensus.cc:1240] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:46.458225 22685 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:46.503090 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.504069 15202 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.504621 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:46.499625 14983 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 15 to 20, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 20 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:46.417692 22682 consensus_queue.cc:1175] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4b633b84351f40de9437e3d532fe6fa7" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 15 index: 13 } }
I20260709 13:05:46.491289 22499 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:46.460152 22747 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:46.540181 15202 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.555658 15202 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.544577 22741 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:46.583119 14983 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 16 to 19, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 19 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.544401 22714 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.509544 22523 consensus_queue.cc:1059] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:46.541760 22742 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:46.559283 15202 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 93 similar messages]
W20260709 13:05:46.593330 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 322821us
W20260709 13:05:46.594269 15128 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.598097 15202 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.599251 15202 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.601215 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.602823 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 311707us
I20260709 13:05:46.605600 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.606171 15840 raft_consensus.cc:3060] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:46.606851 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.607394 15846 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:46.607535 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.609189 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.609750 15848 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:46.613328 15840 raft_consensus.cc:2468] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
W20260709 13:05:46.590453 22565 consensus_meta.cc:287] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.005s
W20260709 13:05:46.615155 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 331026us
W20260709 13:05:46.601927 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.599810 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:46.601682 22769 log.cc:927] Time spent T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
W20260709 13:05:46.628109 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.630045 15201 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.593428 22714 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:46.640718 22767 log.cc:927] Time spent T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:05:46.642158 22762 log.cc:927] Time spent T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:05:46.644039 15813 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.022s
W20260709 13:05:46.644658 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 345755us
I20260709 13:05:46.628455 15848 raft_consensus.cc:2468] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
W20260709 13:05:46.644389 15813 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.000s	sys 0.022s
I20260709 13:05:46.633929 22575 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:46.652721 15201 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.621356 22565 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:46.650892 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 603080us
W20260709 13:05:46.658649 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.648114 22748 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:46.664413 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 337110us
I20260709 13:05:46.654335 22575 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:46.668435 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.657255 22777 raft_consensus.cc:2804] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:46.660495 22750 raft_consensus.cc:2804] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:46.685932 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.657626 22569 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.666046 22508 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:46.702385 15840 raft_consensus.cc:1240] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:05:46.702795 22777 raft_consensus.cc:697] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
W20260709 13:05:46.701877 15813 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.705094 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54628 (request call id 1786) took 1348 ms. Trace:
W20260709 13:05:46.707439 22770 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:05:46.708294 22765 log.cc:927] Time spent T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 13:05:46.708551 22750 raft_consensus.cc:697] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:46.709316 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.713162 22747 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.715502 22778 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:46.719060 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.721801 22752 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.726107 22565 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:46.690896 22715 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.729074 22714 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:46.730800 22527 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.004s
W20260709 13:05:46.734615 15202 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.738162 15846 raft_consensus.cc:2468] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:46.741789 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.686196 22575 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:46.739614 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 46782.8us
I20260709 13:05:46.744830 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.740298 15156 rpcz_store.cc:276] 0709 13:05:45.356493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.356673 (+   180us) service_pool.cc:224] Handling call
0709 13:05:45.391587 (+ 34914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.395636 (+  4049us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:45.989637 (+594001us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b180019afe0. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f58c4bc 00007f0d3c8da48b 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f4455 00007f0d433dc3c7 00007f0d3f9ee9a7 00007f0d3f9fe0bf
0709 13:05:45.989802 (+   165us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.594342 (+604540us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.614308 (+ 19966us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.614500 (+   192us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:46.614537 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.614734 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.701358 (+ 86624us) raft_consensus.cc:1685] finished
0709 13:05:46.704896 (+  3538us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.705030 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19718,"prepare.queue_time_us":49204,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":337076,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":21069,"spinlock_wait_cycles":69164416,"thread_start_us":654194,"threads_started":3,"wal-append.queue_time_us":464590}
W20260709 13:05:46.708336 22773 log.cc:927] Time spent T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:46.692296 22780 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.747537 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.749773 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:46.749853 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 324847us
I20260709 13:05:46.818991 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.743217 22733 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.754621 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.759339 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.744755 22389 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.739714 22755 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.786826 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 137013us
W20260709 13:05:46.786800 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52221.3us
I20260709 13:05:46.789716 15201 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:46.790360 15842 raft_consensus.cc:2368] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21.
I20260709 13:05:46.792207 15231 raft_consensus.cc:2410] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:46.797192 15841 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:46.797753 22353 raft_consensus.cc:3055] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:46.827698 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:46.800086 15157 raft_consensus.cc:1275] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:46.800200 15844 raft_consensus.cc:1240] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:46.768437 22778 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.801198 22750 consensus_queue.cc:237] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.803141 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:46.786962 22715 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.806130 15839 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
W20260709 13:05:46.822108 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 67610.7us
I20260709 13:05:46.839872 15155 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:46.823637 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.824108 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:46.845388 15843 raft_consensus.cc:2368] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:46.827643 15127 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:46.846640 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.833114 22353 raft_consensus.cc:740] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:46.849421 22783 log.cc:927] Time spent T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:05:46.849611 22689 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:46.850317 22689 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.850829 22689 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.832996 22389 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.742833 22508 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.845157 22565 raft_consensus.cc:2804] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:46.853503 22565 raft_consensus.cc:697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:46.853844 22389 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.853322 14983 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.854331 22508 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.854408 22565 consensus_queue.cc:237] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.854980 14983 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.856767 22499 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:46.857247 22499 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:46.849438 22353 consensus_queue.cc:260] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.859870 22353 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:46.860574 22457 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:46.863566 14979 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 370) took 33307 ms (client timeout 15000 ms). Trace:
W20260709 13:05:46.867672 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 44150.3us
W20260709 13:05:46.869680 15810 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.869831 15051 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:46.871459 15202 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.842469 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:46.872467 15202 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.873179 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:46.874516 15202 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.823187 22729 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:46.875205 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.856556 14983 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 15, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 15 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.879469 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.835224 22675 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:46.827032 22649 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:46.898933 15847 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:46.904783 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:46.905045 14983 catalog_manager.cc:5697] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 18, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.906723 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:46.907392 15155 raft_consensus.cc:2468] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:46.907537 15228 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:46.826794 22780 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.910933 14983 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 17 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.911844 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:46.913239 15202 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.912631 14983 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 19, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:46.914173 15202 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.915597 15202 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.916672 15202 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.917449 15202 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:46.919005 15202 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:46.918744 14983 catalog_manager.cc:5697] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.875205 22714 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:46.926376 14983 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.888526 22747 raft_consensus.cc:2804] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:46.937294 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.931699 15156 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:05:46.935231 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 51738.2us
I20260709 13:05:46.825889 22775 consensus_queue.cc:1048] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 13:05:46.891036 22705 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:05:46.834038 22755 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:46.891371 22569 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:46.937391 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 142517us
I20260709 13:05:46.939476 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:46.830418 22733 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.949148 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.950245 15848 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:46.834286 22777 consensus_queue.cc:237] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.952759 14983 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 14 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.982776 14983 catalog_manager.cc:5697] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:46.984140 14983 catalog_manager.cc:5697] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.985419 14983 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.986896 14983 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 20 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:46.953190 22575 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.990052 22575 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:46.954528 22680 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18.
I20260709 13:05:46.955227 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:46.958124 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.991214 15845 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:46.991335 15160 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:46.958803 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.996104 15236 raft_consensus.cc:1275] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:05:46.890843 22457 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:46.897110 22689 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:46.964483 15155 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:46.974874 15203 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:46.977793 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.979907 22359 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:46.918838 22794 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:46.981289 22755 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:46.966770 15051 leader_election.cc:400] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
W20260709 13:05:47.001761 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 62363.2us
I20260709 13:05:47.034701 15051 leader_election.cc:403] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:46.980552 22317 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:46.963994 22790 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:47.014573 15154 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:46.893224 22729 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:46.996132 14983 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 15 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.020128 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:47.020371 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 45580.6us
I20260709 13:05:47.005455 15847 raft_consensus.cc:1240] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:05:47.027789 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:46.960752 22805 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:47.003535 22715 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.004184 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:47.096222 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 73970.3us
I20260709 13:05:47.097106 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:47.098315 15052 leader_election.cc:400] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21.
I20260709 13:05:47.098575 15052 leader_election.cc:403] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:47.005986 22353 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:47.034478 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.084512 15231 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:47.038869 15840 raft_consensus.cc:2468] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:47.107142 22778 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.025753 22688 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:46.997547 22714 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.002812 22685 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.002657 22747 raft_consensus.cc:697] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:47.112902 15163 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:47.115485 15203 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:46.863992 14979 rpcz_store.cc:269] 0709 13:05:13.545619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.545802 (+   183us) service_pool.cc:224] Handling call
0709 13:05:46.852721 (+33306919us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:41.210497 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:41.211415 (+   918us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:41.211499 (+    84us) write_op.cc:435] Acquired schema lock
0709 13:05:41.211525 (+    26us) tablet.cc:662] Decoding operations
0709 13:05:41.429483 (+217958us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:41.553577 (+124094us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:41.553612 (+    35us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:05:41.555391 (+  1779us) tablet.cc:701] Row locks acquired
0709 13:05:41.555434 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:05:41.555606 (+   172us) write_op.cc:270] Start()
0709 13:05:41.555756 (+   150us) write_op.cc:276] Timestamp: P: 1783602341555542 usec, L: 0
0709 13:05:41.555790 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:05:41.559578 (+  3788us) log.cc:844] Serialized 9170 byte log entry
0709 13:05:42.876141 (+1316563us) op_driver.cc:464] REPLICATION: finished
0709 13:05:43.250384 (+374243us) write_op.cc:301] APPLY: starting
0709 13:05:43.250527 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:43.586292 (+335765us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:43.586420 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:44.015010 (+428590us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:44.015813 (+   803us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 13:05:44.015887 (+    74us) write_op.cc:312] APPLY: finished
0709 13:05:44.016623 (+   736us) log.cc:844] Serialized 204 byte log entry
0709 13:05:44.562558 (+545935us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b4400022b4c. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c81dc85 00007f0d3c81e6a2 00007f0d3f5901db 00007f0d3f58baaf 00007f0d3c8da0ac 00007f0d45734ab9 00007f0d3c8f669b 00007f0d3c8f493f 00007f0d3c8f6c02 00007f0d3f57e20b
0709 13:05:44.744449 (+181891us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.744912 (+   463us) write_op.cc:454] Released schema lock
0709 13:05:44.744989 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:44.745634 (+   645us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.745663 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8537456,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":373855,"apply.run_cpu_time_us":61373,"apply.run_wall_time_us":1495268,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100405,"num_ops":23,"prepare.queue_time_us":594187,"prepare.run_cpu_time_us":32102,"prepare.run_wall_time_us":977958,"raft.queue_time_us":67521,"raft.run_cpu_time_us":31560,"raft.run_wall_time_us":413870,"replication_time_us":1320195,"spinlock_wait_cycles":13989248,"thread_start_us":1723520,"threads_started":5,"wal-append.queue_time_us":1613997,"wal-append.run_cpu_time_us":38395,"wal-append.run_wall_time_us":2657643}]]}
I20260709 13:05:47.129750 15160 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:47.116953 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.124027 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.137032 15154 raft_consensus.cc:2393] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:47.125015 15159 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:05:47.109968 22715 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:47.108278 22755 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.086344 22480 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.033869 22499 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.177999 22780 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.178982 22499 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.116986 22794 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:47.145387 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:47.103379 22805 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.172439 14983 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 18 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.114610 22757 raft_consensus.cc:3060] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:47.161810 15156 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:47.143321 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.187819 15159 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
W20260709 13:05:47.164958 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 66789.2us
I20260709 13:05:47.129987 22778 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:47.174502 22747 consensus_queue.cc:237] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.197350 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.199324 14983 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 19, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:47.174973 22685 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.192245 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:47.140344 22688 raft_consensus.cc:2804] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:47.202757 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57487.4us
I20260709 13:05:47.205061 15052 leader_election.cc:400] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:47.205458 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:47.206002 15229 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:47.153035 22689 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:47.117420 22748 consensus_queue.cc:1048] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:47.204277 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:47.214334 22674 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:47.198966 22569 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:47.214911 22499 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:47.163430 22755 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.161764 22800 consensus_queue.cc:1048] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:05:47.198693 22480 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.198264 22715 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 13:05:47.231168 22705 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:47.220768 22688 raft_consensus.cc:697] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:47.249706 22569 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.211295 22778 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.221426 22359 consensus_queue.cc:1059] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
W20260709 13:05:47.265589 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 229061us
I20260709 13:05:47.240103 22674 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:47.241806 22499 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.325551 15229 raft_consensus.cc:2468] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
W20260709 13:05:47.255883 22797 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:05:47.218863 22689 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.332221 15814 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:47.336109 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:47.323101 22662 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.255517 22705 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.332957 15126 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:47.344981 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 378307us
I20260709 13:05:47.350714 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.351022 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.351019 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:47.330928 22499 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:47.367022 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.367807 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.368580 15234 raft_consensus.cc:2393] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:47.368818 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.369213 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.380009 15232 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 13:05:47.380482 15232 pending_rounds.cc:85] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting all ops after (but not including) 13
I20260709 13:05:47.380280 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.380744 15232 pending_rounds.cc:107] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 13:05:47.381079 15232 raft_consensus.cc:2889] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.368949 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.395303 15230 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:47.361946 22359 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:05:47.346371 22730 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:47.346093 22748 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.353724 22685 raft_consensus.cc:2804] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:05:47.358258 22803 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:05:47.384233 22729 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:47.393774 22778 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.416218 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.350068 22695 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:47.350955 22705 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:47.405501 22819 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.407872 22782 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.392912 22649 consensus_queue.cc:1059] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
W20260709 13:05:47.438773 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 84992.8us
I20260709 13:05:47.442641 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.442883 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.420234 22714 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.473932 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.475102 15160 raft_consensus.cc:1275] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:47.392517 22457 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:47.423159 22685 raft_consensus.cc:697] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:47.424827 22695 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:47.475804 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.412326 22748 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.385628 22662 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:47.484462 15128 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:47.485308 15051 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:47.487586 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.491168 15051 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:47.488381 15128 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:47.492887 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:47.496472 15051 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:47.413020 22688 consensus_queue.cc:237] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.501380 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.479755 22457 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.486855 22695 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:47.531803 15230 raft_consensus.cc:2410] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:47.439774 22782 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.506851 22714 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:47.496140 22805 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.502312 22317 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:47.422735 22674 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.538940 15228 raft_consensus.cc:2410] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:05:47.499347 22818 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:47.444142 22819 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.532546 22785 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:47.506709 22755 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:47.489836 22649 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:05:47.540482 22457 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:47.613554 22782 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.614243 22782 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:47.614792 22782 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:47.624100 22821 log.cc:927] Time spent T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:05:47.626016 22820 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:05:47.627758 22822 log.cc:927] Time spent T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 13:05:47.634091 22629 consensus_meta.cc:287] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.218s	user 0.016s	sys 0.000s
W20260709 13:05:47.606518 22823 log.cc:927] Time spent T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:47.607195 22689 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.609938 22819 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.667881 22819 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.668573 22819 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.674264 22662 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.578804 22685 consensus_queue.cc:237] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:47.690543 22794 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:05:47.691201 22794 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21.
I20260709 13:05:47.579094 22714 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.693499 22714 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.696306 22714 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:47.712446 14973 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ff3342a5b2ad49e2b252d7cd47b9851d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d45101e0851484690b4f03a4dcdb63d is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:47.729962 22794 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.730942 22695 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.730722 22794 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.738147 22782 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:47.742699 22757 consensus_meta.cc:287] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.001s
I20260709 13:05:47.746325 22819 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.777392 22499 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:47.779066 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 277163us
W20260709 13:05:47.840649 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:47.841547 15128 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:47.789209 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:47.854147 22794 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.834270 22757 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for earlier term 17. Current term is 18.
I20260709 13:05:47.844241 15159 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:47.857079 22675 raft_consensus.cc:3055] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:47.857627 22675 raft_consensus.cc:740] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:47.858583 22675 consensus_queue.cc:260] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.860281 22675 raft_consensus.cc:3060] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:47.831375 22714 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.870325 15844 raft_consensus.cc:1275] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:47.755923 22705 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.815979 22755 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.817080 22777 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:47.778494 22690 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.801080 22472 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.838701 15839 raft_consensus.cc:2393] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:47.830626 22457 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.908684 15233 raft_consensus.cc:1240] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:47.873412 22714 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.891093 22700 consensus_queue.cc:1048] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 13:05:47.922559 15128 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:47.925334 22733 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.926057 22700 consensus_queue.cc:1048] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:47.891134 22472 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.936004 22700 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:47.936563 22700 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.937577 22480 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.937674 22472 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.889787 22777 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.938880 22700 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.915385 22688 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:47.940222 22777 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.888155 22755 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.941131 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.941771 15236 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:47.942118 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.942587 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.945201 15236 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:47.890235 22690 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.946913 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.947122 22690 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:47.942953 22755 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.920449 22685 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.948112 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.948011 22685 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:47.935349 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:47.944355 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.934945 22695 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.948813 15231 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:47.949815 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2832) took 1030 ms. Trace:
I20260709 13:05:47.951635 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:47.940658 22688 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.965608 15847 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:47.967571 22688 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.971505 22480 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.972266 22480 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:47.964555 15155 rpcz_store.cc:276] 0709 13:05:46.919036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.919602 (+   566us) service_pool.cc:224] Handling call
0709 13:05:46.937184 (+ 17582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.006283 (+ 69099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.949593 (+943310us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b540033e354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:47.949758 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62038912}
I20260709 13:05:47.975303 22480 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.976085 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:47.993131 22688 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:47.993857 22688 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.994359 22688 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:47.951952 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:48.044471 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 92587.8us
W20260709 13:05:48.045734 15128 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.047469 15128 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:47.941138 22714 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.950119 22685 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:47.951027 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:48.090155 15230 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:47.951313 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:48.091568 15236 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:48.092012 22733 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.092614 22733 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.095362 22733 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:47.953701 15843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59152 (request call id 1748) took 1097 ms. Trace:
I20260709 13:05:48.078747 22829 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.083204 22781 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.113533 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59122 (request call id 1822) took 1230 ms. Trace:
I20260709 13:05:48.104681 15843 rpcz_store.cc:276] 0709 13:05:46.856613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.861073 (+  4460us) service_pool.cc:224] Handling call
0709 13:05:46.861394 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.112594 (+251200us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:47.136723 (+ 24129us) log.cc:844] Serialized 14 byte log entry
0709 13:05:47.420302 (+283579us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:47.420381 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:47.420457 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.952606 (+532149us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b180040f840. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8c52a6 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419
0709 13:05:47.953458 (+   852us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54004a2754. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:47.953511 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.953643 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17352,"prepare.queue_time_us":27289,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2882,"raft.queue_time_us":113,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2996,"spinlock_wait_cycles":586769536,"thread_start_us":283049,"threads_started":1,"wal-append.queue_time_us":366029}
W20260709 13:05:48.111534 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64097.9us
I20260709 13:05:48.103322 22677 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.118320 15128 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.118930 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:48.121603 15236 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:48.122956 22781 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:48.129038 15128 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.130302 15128 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.124989 22677 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.114099 15842 rpcz_store.cc:276] 0709 13:05:46.883119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.883282 (+   163us) service_pool.cc:224] Handling call
0709 13:05:46.897997 (+ 14715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.898232 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:47.026054 (+127822us) log.cc:844] Serialized 14 byte log entry
0709 13:05:47.027127 (+  1073us) log.cc:844] Serialized 14 byte log entry
0709 13:05:47.955616 (+928489us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800439a20. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c871c0a 00007f0d45fb1219 00007f0d3f59ad70 00007f0d3f58e293 00007f0d3f583cea 00007f0d3f58423a 00007f0d3f9eea2a 00007f0d3f9fe0bf 00007f0d3f9fe038
0709 13:05:48.113012 (+157396us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:48.113102 (+    90us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:48.113172 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.113266 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.113445 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184189,"prepare.queue_time_us":184515,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":11690,"raft.queue_time_us":7716,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":54112,"spinlock_wait_cycles":25414400,"thread_start_us":104369,"threads_started":1}
W20260709 13:05:48.140355 15128 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.112655 22793 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:48.131717 22480 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:48.150262 22784 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:05:48.149901 15202 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.157259 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:48.139147 22742 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.160637 22742 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.161334 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59122 (request call id 1821) took 1426 ms. Trace:
I20260709 13:05:48.161689 15840 rpcz_store.cc:276] 0709 13:05:46.734464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.735037 (+   573us) service_pool.cc:224] Handling call
0709 13:05:48.161148 (+1426111us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400498c54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:48.161257 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400670976}
I20260709 13:05:48.147228 22695 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.150820 22781 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.152700 22527 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.155913 22705 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.127678 22829 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.151556 22457 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:48.209661 22830 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:05:48.211690 22742 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.152055 22793 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:48.197376 22781 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.183627 22527 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.213090 22781 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.215653 22527 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:48.227918 15128 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.232954 15128 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.234094 15128 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.235677 15128 leader_election.cc:336] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.171465 22695 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.158975 22784 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:48.272805 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2839) took 1128 ms. Trace:
I20260709 13:05:48.274021 15156 rpcz_store.cc:276] 0709 13:05:47.109225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.109377 (+   152us) service_pool.cc:224] Handling call
0709 13:05:48.237595 (+1128218us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b540006f254. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:48.237797 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58482560}
W20260709 13:05:48.270953 15128 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.231199 22480 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.230553 22780 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:48.291270 22813 log.cc:927] Time spent T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 1.039s	user 0.008s	sys 0.005s
I20260709 13:05:48.271714 22682 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.231796 22499 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.237949 22742 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.283421 22781 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:48.302862 15128 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.293414 22480 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:48.296135 22780 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.315274 22480 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.317685 22695 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.322602 22695 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.323359 22695 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.329491 22695 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.298470 22682 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.315352 22780 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.310323 22677 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.302435 22742 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.428663 22781 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.442891 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.338749 22829 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.497910 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.504161 15233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2837) took 1096 ms. Trace:
I20260709 13:05:48.505820 15233 rpcz_store.cc:276] 0709 13:05:47.368318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.370145 (+  1827us) service_pool.cc:224] Handling call
0709 13:05:47.370480 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.912002 (+541522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.464166 (+552164us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54002d4454. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:48.464413 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237134080}
I20260709 13:05:48.508435 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.457778 22569 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.497195 22674 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.497668 22748 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.496932 22480 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.536665 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.538537 15201 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:48.507534 22800 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
W20260709 13:05:48.535535 22688 consensus_meta.cc:287] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.004s
I20260709 13:05:48.547271 15840 raft_consensus.cc:2393] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:48.535744 22793 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.536865 22569 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.542501 22677 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.513840 22781 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.548749 22829 raft_consensus.cc:2804] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:48.545871 22784 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.560381 22677 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.543606 22674 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.505646 22675 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.546698 22480 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.560029 22728 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.583456 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 417726us
I20260709 13:05:48.583961 15842 raft_consensus.cc:1275] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:48.550961 22688 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.585245 15231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2842) took 1135 ms. Trace:
I20260709 13:05:48.555015 22800 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.575343 22829 raft_consensus.cc:697] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:48.591840 22682 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.587248 22728 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.597903 15231 rpcz_store.cc:276] 0709 13:05:47.449744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.962114 (+512370us) service_pool.cc:224] Handling call
0709 13:05:47.962531 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.963295 (+   764us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:48.219025 (+255730us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.584699 (+365674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:48.584760 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:48.584829 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.584965 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.585144 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2128,"prepare.queue_time_us":567847,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":2268,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2679,"thread_start_us":617581,"threads_started":2}
I20260709 13:05:48.602849 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.596581 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:48.619567 22742 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.651149 15233 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:48.582891 22675 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:48.672940 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 45406.5us
I20260709 13:05:48.677335 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.677587 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.680653 15159 raft_consensus.cc:2410] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:48.624688 22748 consensus_queue.cc:1048] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
W20260709 13:05:48.685072 15811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.026s
W20260709 13:05:48.687261 15811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.026s
I20260709 13:05:48.687924 22781 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.709820 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:48.676304 22499 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.713562 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:48.710855 15233 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
I20260709 13:05:48.714774 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.716485 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.717335 15846 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:48.712505 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:48.683246 22779 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.711318 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.721462 15844 raft_consensus.cc:3060] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:48.721657 15839 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:48.723749 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.724586 15839 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:48.725754 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.726333 22480 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.726420 15845 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 17.
I20260709 13:05:48.722663 22508 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:48.729236 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.729566 22508 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.730367 15847 raft_consensus.cc:2393] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:48.732825 22508 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.733721 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:48.733971 15813 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.776s (SENT)
I20260709 13:05:48.743768 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:48.746587 22389 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.747311 22389 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.750514 22389 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:48.752441 15813 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Timed out: RequestConsensusVote RPC to 127.13.163.193:38845 timed out after 1.796s (SENT)
I20260709 13:05:48.753124 15813 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:48.757688 22728 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.758396 22389 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:48.760066 22728 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.711378 22682 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:48.760802 22728 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.761150 22682 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.764087 22728 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:48.684355 22829 consensus_queue.cc:237] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.769479 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.699851 22777 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.777207 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.771972 15842 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 22.
I20260709 13:05:48.700872 22569 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.700654 22677 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.785668 15841 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:48.786176 15228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43960 (request call id 2833) took 1416 ms. Trace:
I20260709 13:05:48.786777 15228 rpcz_store.cc:276] 0709 13:05:47.367188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.367304 (+   116us) service_pool.cc:224] Handling call
0709 13:05:48.783655 (+1416351us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002148d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:48.783868 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291218176}
I20260709 13:05:48.787273 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:48.782366 22849 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.129s	user 0.001s	sys 0.001s
I20260709 13:05:48.808239 22816 raft_consensus.cc:3055] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:48.808773 22816 raft_consensus.cc:740] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:48.797921 15844 raft_consensus.cc:2468] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 22.
I20260709 13:05:48.810482 22816 consensus_queue.cc:260] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.813522 22800 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.764104 22682 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:48.815965 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.816732 15162 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:48.819077 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.820106 15160 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:48.821652 22688 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.770854 22775 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.758570 22499 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.826098 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.825220 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:48.833245 15157 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:48.834406 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:48.825814 15813 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.829524 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.824257 15232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1714) took 1048 ms. Trace:
I20260709 13:05:48.783741 22784 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:48.837805 15232 rpcz_store.cc:276] 0709 13:05:47.775533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.098102 (+322569us) service_pool.cc:224] Handling call
0709 13:05:48.823584 (+725482us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:48.823784 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.823951 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.823979 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:48.824026 (+    47us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:48.824054 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.824109 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.824207 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26271488}
I20260709 13:05:48.832459 15154 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 22.
I20260709 13:05:48.841624 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.844053 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.849092 15155 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
I20260709 13:05:48.835207 15233 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 15.
I20260709 13:05:48.837363 15161 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:48.861538 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.861661 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.890425 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:48.860895 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.853449 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.891047 15841 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:48.892360 15843 raft_consensus.cc:2393] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:48.891654 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.893692 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.898671 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.799683 22677 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.898689 15160 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 16.
W20260709 13:05:48.874161 15813 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:48.905066 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53287.2us
I20260709 13:05:48.770629 22779 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.908601 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.910259 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:48.843928 22800 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.912375 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:48.902256 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:48.913937 15845 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:48.831038 22674 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.785228 22777 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.927816 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:48.922137 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.930899 15843 raft_consensus.cc:2393] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:48.924342 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.933800 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.934026 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:48.934701 15154 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:48.925096 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.891415 22690 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.928576 15813 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.889849 22682 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:48.930742 15163 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:48.948557 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.949280 15163 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:05:48.923502 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:48.890883 15839 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 20.
I20260709 13:05:48.900378 22569 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.901051 22816 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:48.936545 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.921097 15814 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:48.938510 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.966396 15162 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:48.939795 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:48.966713 15161 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
W20260709 13:05:48.939764 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 261255us
I20260709 13:05:48.964733 15845 raft_consensus.cc:1240] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:48.845428 22775 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:48.968993 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:48.969856 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:48.972570 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 51559.9us
W20260709 13:05:48.973165 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44623.9us
W20260709 13:05:48.974854 15813 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.972414 15163 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:48.901302 22784 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.975333 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:48.919579 22800 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.946426 22742 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:48.967273 22845 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:48.916425 22677 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.970999 22520 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:48.970325 22675 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.971179 22780 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:48.992134 15158 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:48.978276 22729 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:48.985314 15813 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:48.997815 22800 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:48.999905 22861 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:48.994179 22674 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.002027 22674 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.003340 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.004119 22861 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:48.994366 22793 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.004930 22793 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.005391 15054 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.006117 22861 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:49.006907 22793 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.007905 22793 consensus_queue.cc:1059] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:49.008776 22861 raft_consensus.cc:3055] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:49.009131 22861 raft_consensus.cc:740] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:49.009922 22861 consensus_queue.cc:260] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.011274 22861 raft_consensus.cc:3060] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:49.012390 22499 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.013510 22389 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.019320 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.021792 22861 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:49.023242 22861 raft_consensus.cc:3060] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:48.990250 22742 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.030386 22777 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.004177 22674 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.038179 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1713) took 1263 ms. Trace:
I20260709 13:05:49.042346 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1718) took 1181 ms. Trace:
I20260709 13:05:49.038756 15229 rpcz_store.cc:276] 0709 13:05:47.774908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.093357 (+318449us) service_pool.cc:224] Handling call
0709 13:05:49.037185 (+943828us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800165698. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3c77fdb1 00007f0d3c8bfb71 00007f0d3c8bfa22 00007f0d457d8c09 00007f0d44932ee0 00007f0d449fc0ab 00007f0d449fc131 00007f0d449bdfc0 00007f0d449a8929 00007f0d449a83d2 00007f0d3df1fbd4
0709 13:05:49.037479 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.037739 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.037779 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.037820 (+    41us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:49.037862 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.037916 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.038095 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39732992}
I20260709 13:05:48.967697 22690 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.028317 22780 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.042982 15235 rpcz_store.cc:276] 0709 13:05:47.861268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.211453 (+350185us) service_pool.cc:224] Handling call
0709 13:05:48.211771 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.211981 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:48.696436 (+484455us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.041912 (+345476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.041982 (+    70us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:49.042042 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.042130 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.042276 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343815,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1199,"raft.queue_time_us":108,"thread_start_us":825355,"threads_started":2}
I20260709 13:05:48.968907 22682 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:48.971802 22569 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.059916 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:48.995859 22729 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.064077 22682 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:49.065816 22569 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.066534 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.083148 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.069180 22792 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:49.076289 22677 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.077169 22779 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.048041 22508 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.081199 15848 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:49.103658 15157 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
I20260709 13:05:49.108439 22815 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:49.108953 22815 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.109274 22815 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:49.083781 22389 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:49.118034 22389 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.118247 22690 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.118454 22389 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.103401 22792 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.119139 22792 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.121985 22780 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.081264 22729 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.130183 15203 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.162976 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.170212 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.172468 15203 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.174443 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.161104 22828 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:49.170691 22777 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:49.185817 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 188578us
I20260709 13:05:49.191598 15841 raft_consensus.cc:2468] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:49.172055 22748 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.202553 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.192830 22828 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.226904 22845 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.227409 22845 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.227488 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.228312 15232 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 22.
W20260709 13:05:49.229267 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.230989 22861 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:49.234099 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.234961 22523 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:49.235409 22523 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.235822 22523 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:49.227543 22828 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.236347 22845 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.224076 22750 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.239570 22845 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:49.244433 22523 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.194160 22777 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.224803 22748 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:49.250429 22741 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:49.224531 22775 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.250702 22784 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:49.281754 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 303630us
W20260709 13:05:49.281044 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 261791us
I20260709 13:05:49.285842 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:49.293267 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.293316 15232 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:49.293828 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.294216 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.294765 15052 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.290553 15127 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.276645 22748 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:49.269111 22777 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.297200 15127 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.284318 22523 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.279942 22742 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.301460 15841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.322341 15127 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:49.303016 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.306618 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.324536 15843 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:49.326385 15127 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.327728 15127 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:49.308723 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.320685 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.303714 15201 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.322499 15841 raft_consensus.cc:2393] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 23: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:49.279196 22741 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:49.332710 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.337881 15127 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.326385 22674 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:05:49.341178 22862 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:49.341706 22862 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:49.342097 22862 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:49.343798 22857 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:49.337734 22741 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.350219 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.350577 22862 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.352723 22862 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:49.354573 22858 log.cc:927] Time spent T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:49.354821 22857 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:49.339708 22569 raft_consensus.cc:3060] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.357673 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:49.358408 15848 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:49.366182 15848 raft_consensus.cc:2468] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 22.
I20260709 13:05:49.367085 22800 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:49.367754 22800 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.306923 22784 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:49.298105 22775 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.306574 22748 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:49.326431 22861 raft_consensus.cc:3060] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:49.292959 22389 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.328554 22780 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:49.349741 22859 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:05:49.344280 22815 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.337424 22798 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.338495 22499 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:49.390625 14973 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ff3342a5b2ad49e2b252d7cd47b9851d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d45101e0851484690b4f03a4dcdb63d is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:49.339888 22779 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:49.339653 22742 raft_consensus.cc:2804] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:49.427137 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 252723us
I20260709 13:05:49.429672 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:49.412424 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 433254us
I20260709 13:05:49.433894 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:49.436918 15232 raft_consensus.cc:2468] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 23.
I20260709 13:05:49.434724 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.396229 22780 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:49.438731 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:49.451807 15233 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:49.445225 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.406734 22499 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.432626 15051 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.471829 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.408442 22779 raft_consensus.cc:493] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:49.446556 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.449231 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.445832 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.473600 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.459085 15201 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.474005 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.474723 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.375023 22784 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.478278 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.452299 15810 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.475422 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.473600 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.483031 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.480367 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:49.421049 22793 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.485163 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.486371 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.487521 15230 raft_consensus.cc:2393] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 17: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:49.490551 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.411926 22867 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.425450 22742 raft_consensus.cc:697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:49.493911 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:49.496244 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:49.495816 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:49.495292 15163 raft_consensus.cc:1275] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:05:49.493590 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:49.509025 15228 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 19: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:49.492942 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:49.509337 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.500809 15233 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:49.499267 15236 raft_consensus.cc:1240] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:49.500484 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 48258.6us
I20260709 13:05:49.438526 22565 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.496642 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.521706 15814 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.512490 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.473690 22842 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:49.513413 22695 raft_consensus.cc:2804] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:49.477515 22779 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:49.514622 15232 raft_consensus.cc:1275] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:49.409132 22792 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.523669 15841 raft_consensus.cc:1275] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:49.478504 22499 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:49.471601 22780 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:49.513681 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:49.510231 15202 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:49.548712 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.509186 22848 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:49.488569 22865 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:49.507323 22677 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:49.525882 22842 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.500304 22868 raft_consensus.cc:2804] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:49.533121 22730 raft_consensus.cc:2804] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:49.527776 22871 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:49.581943 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 222827us
I20260709 13:05:49.579499 15228 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:49.528877 22682 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:49.555016 22695 raft_consensus.cc:697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:49.586661 22695 consensus_queue.cc:237] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.501747 22793 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.588272 22868 raft_consensus.cc:697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:49.590128 22793 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.589639 22868 consensus_queue.cc:237] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.592921 22682 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.594393 22682 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.555642 22848 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.579398 22842 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.552222 22862 consensus_queue.cc:1048] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:05:49.558051 22865 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.573359 22508 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:49.615039 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 264011us
I20260709 13:05:49.561730 22457 consensus_queue.cc:1059] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:49.618468 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:49.627673 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 105043us
I20260709 13:05:49.628468 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:49.642937 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.608444 22848 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:49.604519 22730 raft_consensus.cc:697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:49.659426 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.667217 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.668831 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.617524 22865 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.657773 22793 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:05:49.604135 22742 consensus_queue.cc:237] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.658629 22872 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:49.684631 22792 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.649194 22674 consensus_queue.cc:1059] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:05:49.667075 22389 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.669414 22728 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.702169 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.702683 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 153247us
W20260709 13:05:49.703318 15202 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:49.704159 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.706561 22800 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.711644 22779 raft_consensus.cc:515] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:49.713447 22874 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:05:49.714650 22876 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:49.715301 22876 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.716958 22779 leader_election.cc:290] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:49.719507 22842 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.724080 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:49.724751 15230 raft_consensus.cc:3060] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.725958 15231 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:49.676079 22457 raft_consensus.cc:3060] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.729183 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:49.729808 15233 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.733304 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:49.733903 22890 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:49.734004 15228 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:05:49.704663 15202 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:49.738359 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.745000 15230 raft_consensus.cc:2468] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:49.676348 22815 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:49.754077 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 49439.8us
I20260709 13:05:49.750305 22674 consensus_queue.cc:1243] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:05:49.758988 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:49.762728 15233 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:49.756697 15810 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.765674 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.694487 22784 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:49.692951 22872 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.751571 22793 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:49.810920 15810 leader_election.cc:304] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.811908 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:49.812549 15163 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.812402 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:49.776832 22828 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:49.763849 22890 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.706620 22508 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.781255 22674 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.764353 22868 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:49.754871 22730 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.795758 22842 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:49.802888 22872 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:49.790043 22691 raft_consensus.cc:2804] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:49.755940 22728 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:49.802263 22729 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.672s	user 0.008s	sys 0.004s
W20260709 13:05:49.808475 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 313129us
W20260709 13:05:49.845649 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 176892us
I20260709 13:05:49.845749 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.809615 15842 raft_consensus.cc:1275] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:49.814137 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:49.846671 15155 raft_consensus.cc:3060] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:49.814548 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:49.847465 15159 raft_consensus.cc:3060] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:49.815094 15228 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
W20260709 13:05:49.817395 15128 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:49.823840 15202 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:49.828282 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 209872us
I20260709 13:05:49.850378 15202 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.832351 22800 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:49.869875 22800 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.870817 22457 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:49.832969 22775 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:49.833442 22861 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:49.835959 22708 raft_consensus.cc:3055] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:49.829967 22508 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:49.827965 22569 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:49.784629 22777 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:49.821133 22677 raft_consensus.cc:2804] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:49.842314 22741 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.843080 22691 raft_consensus.cc:697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:49.843458 22674 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:49.832463 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 116895us
I20260709 13:05:49.850672 15127 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:49.827749 22781 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.846879 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:49.855134 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.840583 22752 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:49.869271 22728 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:49.888298 15163 raft_consensus.cc:2468] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:49.889254 15810 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:49.900049 15159 raft_consensus.cc:2468] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:49.872735 22857 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:49.875092 22775 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.957666 15231 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:49.908788 15155 raft_consensus.cc:2468] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 17.
W20260709 13:05:49.913244 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62656.6us
I20260709 13:05:49.926963 22861 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:49.914780 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.916461 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.875442 22868 consensus_queue.cc:1048] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:49.918614 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:49.919618 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 72791.3us
W20260709 13:05:49.919862 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 64831.3us
I20260709 13:05:49.885170 22708 raft_consensus.cc:740] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:49.928699 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 79307.1us
I20260709 13:05:49.928889 15229 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:49.872119 22729 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:49.924079 15813 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:49.959196 15127 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:50.039825 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 80736.7us
I20260709 13:05:49.913764 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:49.917755 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.873931 22848 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.959746 22861 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.963315 15842 raft_consensus.cc:2410] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:05:49.989306 15126 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:49.926901 22389 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:49.925021 22499 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:50.023331 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 100678us
I20260709 13:05:49.960563 22775 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:50.033051 15128 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:49.934388 22752 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.037489 22728 raft_consensus.cc:2804] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:49.844671 22890 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:49.960649 22677 raft_consensus.cc:697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:50.039348 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.073697 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.047627 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.050738 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:50.052065 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 62816.1us
I20260709 13:05:50.076433 15839 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:50.076809 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.050269 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.075029 15156 raft_consensus.cc:1240] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: []
W20260709 13:05:50.048898 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 225078us
I20260709 13:05:50.057907 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.057737 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.078451 15157 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:49.928428 22480 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:49.928658 22781 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.086088 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.069881 22800 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.004402 22876 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.031178 22828 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.071776 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:49.933813 22682 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.074321 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.078617 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.080814 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
W20260709 13:05:50.082067 15202 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.139140 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:49.874554 22884 consensus_queue.cc:1059] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:05:50.084887 22871 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:50.085711 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:50.094620 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 180098us
I20260709 13:05:50.063514 22861 raft_consensus.cc:3060] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.052259 22857 raft_consensus.cc:2804] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:50.089402 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.152720 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:50.091516 15162 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:50.141539 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:50.037025 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 147871us
W20260709 13:05:50.141140 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 57922.8us
I20260709 13:05:50.153914 15157 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:50.154752 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.157436 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.157816 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:50.158947 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 76907.2us
I20260709 13:05:50.062570 22885 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:50.162407 15841 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
W20260709 13:05:50.142571 15813 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.073748 22389 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.143040 22800 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.135090 22728 raft_consensus.cc:697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:50.186264 15161 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:50.191936 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 115187us
I20260709 13:05:50.151984 22854 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:50.175539 22798 consensus_meta.cc:287] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.002s
I20260709 13:05:50.194043 15236 raft_consensus.cc:1240] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:50.196267 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:50.175997 15813 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.201658 15843 raft_consensus.cc:2468] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:50.083382 22708 consensus_queue.cc:260] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.187321 22750 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.166067 22869 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:50.187500 22884 raft_consensus.cc:3055] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:05:50.208097 15813 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:50.215602 15813 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.215896 15126 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.073359 22691 consensus_queue.cc:237] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.205745 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.202488 22777 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:05:50.259047 15813 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:50.269841 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 54075us
W20260709 13:05:50.217409 22748 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.823s	user 0.007s	sys 0.004s
I20260709 13:05:50.211122 22793 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.243206 22885 consensus_queue.cc:1048] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:05:50.204907 22798 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.193823 22677 consensus_queue.cc:237] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.270557 22728 consensus_queue.cc:237] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.217178 22911 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.271261 15847 raft_consensus.cc:1240] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:50.179768 22865 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.209429 22862 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.274616 22904 log.cc:927] Time spent T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:05:50.213809 22741 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.268793 22674 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.372504 15814 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:50.376868 22862 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.327116 22848 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.286118 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:50.288401 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 131036us
W20260709 13:05:50.382537 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 96491.2us
I20260709 13:05:50.382725 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.272166 22871 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.273813 22729 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.214903 22884 raft_consensus.cc:740] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:50.276780 22869 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.331408 22816 raft_consensus.cc:3055] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:05:50.214282 22457 consensus_queue.cc:1175] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 22 status { last_received { term: 21 index: 13 } last_committed_idx: 13 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:05:50.288782 22894 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:05:50.336021 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.293329 22748 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:50.325564 22900 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 13:05:50.285727 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.383808 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.359229 22800 raft_consensus.cc:493] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:50.289578 22714 consensus_queue.cc:1059] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 13:05:50.368470 22777 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.396157 22389 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.398375 22884 consensus_queue.cc:260] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.399878 22890 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:50.400454 22902 log.cc:927] Time spent T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:05:50.385128 22682 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.412393 22865 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.393561 22871 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.393050 22520 raft_consensus.cc:2804] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:05:50.410203 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:50.440068 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44650.7us
I20260709 13:05:50.368672 22910 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:50.441025 15813 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.443159 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.431348 22913 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:50.423576 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 137937us
I20260709 13:05:50.415402 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.385473 22781 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.427500 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.390787 22674 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:50.390923 22798 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:50.388463 22752 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.437836 15235 raft_consensus.cc:1240] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:50.392824 22499 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.412608 22816 raft_consensus.cc:740] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:50.432253 22885 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:50.447170 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.418022 22857 raft_consensus.cc:697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
W20260709 13:05:50.448323 15813 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.379525 22911 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:50.481583 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.481781 15813 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.450044 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.449210 22520 raft_consensus.cc:697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:50.485390 22775 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.485913 22520 consensus_queue.cc:237] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.487627 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:50.488185 22674 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:50.488291 15159 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.463887 22816 consensus_queue.cc:260] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.489454 22816 raft_consensus.cc:3060] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:50.434185 22777 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.517832 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:50.518699 15157 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:50.523039 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:50.446233 22884 raft_consensus.cc:3060] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:50.470669 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:50.474433 22871 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:50.477320 22508 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.579s	user 0.005s	sys 0.005s
I20260709 13:05:50.445858 22910 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.479943 22748 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.432814 22800 raft_consensus.cc:515] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.481623 15054 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.519502 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.588791 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:50.585763 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.585953 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.589445 15237 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.586489 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.456924 22792 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:50.585469 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:50.517717 22780 consensus_meta.cc:287] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.959s	user 0.011s	sys 0.001s
I20260709 13:05:50.480801 22888 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.571102 15847 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:50.592365 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.599109 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:50.607340 15157 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 19: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:50.605995 15158 raft_consensus.cc:2468] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 17.
I20260709 13:05:50.631577 15163 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:50.561372 22682 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.585019 22695 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.534525 22775 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.635375 15158 raft_consensus.cc:1240] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:50.591557 22508 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.637653 15157 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:05:50.598232 22871 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.617416 22780 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.679286 15842 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:50.679443 15161 raft_consensus.cc:3060] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.671555 22695 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:50.689940 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.557662 22857 consensus_queue.cc:237] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.727730 22457 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.728590 22457 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:50.633065 22888 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.741343 22780 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:50.766333 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 318014us
W20260709 13:05:50.781206 15813 leader_election.cc:336] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:50.776986 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 295435us
I20260709 13:05:50.783710 15054 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.777815 15848 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:05:50.774127 22911 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:50.786284 15813 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.786942 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.788951 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.793622 15054 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.794642 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.800771 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.802042 15847 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:50.782605 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.803735 15848 raft_consensus.cc:2393] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 22: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:50.805120 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:50.809319 15813 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.810349 15159 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:50.811087 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.811298 15163 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:50.817673 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.819770 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.820817 15845 raft_consensus.cc:2368] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
W20260709 13:05:50.821702 15813 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.822396 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.847735 15161 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:50.851907 15228 raft_consensus.cc:1275] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:50.776858 22499 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:50.865999 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.866806 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:50.862892 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:50.867522 15847 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:50.868219 15127 leader_election.cc:400] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:50.868574 15127 leader_election.cc:403] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:05:50.858238 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.859627 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:50.819924 22828 consensus_meta.cc:287] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.006s
I20260709 13:05:50.876551 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:50.877724 15157 raft_consensus.cc:2393] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in current term 23: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:50.871577 15237 raft_consensus.cc:2468] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:50.872584 15814 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:50.816192 22867 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.008s	sys 0.008s
I20260709 13:05:50.814601 22913 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.819983 22389 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.815557 22871 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.837903 22682 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.832247 22865 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:50.831921 22714 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:50.889739 22867 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.009s	sys 0.008s
W20260709 13:05:50.920185 14973 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ff3342a5b2ad49e2b252d7cd47b9851d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d45101e0851484690b4f03a4dcdb63d is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:50.929114 22913 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:50.929805 22913 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:50.930423 22800 leader_election.cc:290] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.931952 22913 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:50.899109 22457 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.933241 22888 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.933521 22457 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:50.899863 22828 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.941125 22862 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:50.941800 15154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.076s	sys 0.099s
I20260709 13:05:50.865329 22508 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.878895 22890 raft_consensus.cc:2804] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:50.840972 22845 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
W20260709 13:05:50.853622 22691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.002s	sys 0.019s
I20260709 13:05:50.952910 22695 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:50.954743 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 131716us
I20260709 13:05:50.957103 22910 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:50.957319 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:50.949584 22688 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:50.932534 22913 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:05:50.961678 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec09a7a4d59c4296aa02dde6bd37f81c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:50.984350 15228 raft_consensus.cc:2368] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:50.947638 22890 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:50.962535 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:50.909907 22677 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:50.893692 22781 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:50.964995 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:50.965191 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:50.972729 22861 consensus_meta.cc:287] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.811s	user 0.022s	sys 0.028s
I20260709 13:05:50.989661 15234 raft_consensus.cc:2368] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:50.967495 15203 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:50.977293 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:50.963387 22730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.007s	sys 0.033s
I20260709 13:05:50.893177 22499 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:50.981060 15813 leader_election.cc:336] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:50.963120 22862 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:50.961988 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:50.995730 15813 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.004865 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.006039 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "639c51e1c4da436b99c010fb41cebc63" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.007844 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.009196 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:51.011417 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:50.970371 22691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.002s	sys 0.019s
W20260709 13:05:51.032274 13967 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:51.033324 15230 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:51.033691 15228 raft_consensus.cc:2393] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 18: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:51.033833 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 76611.7us
I20260709 13:05:51.037334 15157 raft_consensus.cc:3060] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:51.038283 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.039757 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:51.041191 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:51.042497 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.043358 15163 raft_consensus.cc:2393] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:51.044255 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:51.048060 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.048622 15159 raft_consensus.cc:2468] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:50.978065 22828 consensus_queue.cc:1048] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:05:50.982846 22777 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:50.988445 22890 raft_consensus.cc:3060] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:51.058224 22872 consensus_meta.cc:287] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 1.184s	user 0.032s	sys 0.028s
I20260709 13:05:51.058660 22872 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.059791 15840 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:51.061825 22872 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:51.065943 15203 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:51.067610 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1741) took 1614 ms. Trace:
I20260709 13:05:51.068063 15235 rpcz_store.cc:276] 0709 13:05:49.453081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.505084 (+ 52003us) service_pool.cc:224] Handling call
0709 13:05:49.505414 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.456858 (+951444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.067391 (+610533us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b5400177354. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:51.067556 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2139601280}
I20260709 13:05:51.071825 15126 leader_election.cc:400] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:51.073130 15126 leader_election.cc:403] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:51.000126 22829 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:51.094648 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.107533 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.108767 15126 leader_election.cc:400] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:51.110245 15126 leader_election.cc:304] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:51.111094 22569 consensus_queue.cc:1059] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:51.111205 22779 raft_consensus.cc:2749] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 17. Current term is 18.
I20260709 13:05:51.052409 22910 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:50.987159 22688 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:51.113798 15051 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:51.117244 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:51.120741 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 54883.9us
I20260709 13:05:51.149855 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:51.154372 15157 raft_consensus.cc:1275] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:51.000856 22861 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.175204 22861 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.095227 22752 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:51.156569 22903 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.262s
W20260709 13:05:51.177647 22928 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:05:51.178062 22569 raft_consensus.cc:3055] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:51.178705 22569 raft_consensus.cc:740] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:51.146433 22922 raft_consensus.cc:2749] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:51.179924 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.179770 22569 consensus_queue.cc:260] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:51.184022 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 70317.4us
I20260709 13:05:51.168249 15203 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:51.185719 15051 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:51.156832 22918 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.261s
I20260709 13:05:51.170104 22499 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:51.175225 22884 consensus_meta.cc:287] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.007s
I20260709 13:05:51.164259 22688 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:51.034190 22862 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:51.192478 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.193853 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.196221 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:51.185441 22728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.023s	sys 0.033s
I20260709 13:05:51.201328 15161 raft_consensus.cc:1275] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:05:51.183454 22816 consensus_meta.cc:287] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.694s	user 0.019s	sys 0.006s
I20260709 13:05:51.185380 22829 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:51.205271 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.187873 22714 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:51.209316 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:51.191614 22861 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:51.214120 15235 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 3c28be6f86ea4fb6b79a47d3999f5a82 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:51.191334 22752 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:51.223043 22869 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: Time spent flushing consensus metadata: real 0.802s	user 0.004s	sys 0.017s
I20260709 13:05:51.223531 22869 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:51.216315 22897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.021s	sys 0.025s
W20260709 13:05:51.225730 22897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.021s	sys 0.026s
I20260709 13:05:51.229820 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.202382 22569 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:51.216696 22861 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.238196 22861 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:51.239365 22748 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.239956 22748 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.242074 15155 raft_consensus.cc:2468] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:51.243108 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.247315 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:51.204074 22911 consensus_queue.cc:1048] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:05:51.259466 22857 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:51.261343 22926 log.cc:927] Time spent T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:05:51.262248 22748 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.267177 22793 raft_consensus.cc:3055] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:51.267654 22793 raft_consensus.cc:740] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:51.259554 22457 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:51.274856 15810 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:51.288766 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 429224us
W20260709 13:05:51.273952 22923 log.cc:927] Time spent T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:05:51.268477 22793 consensus_queue.cc:260] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.291832 22691 raft_consensus.cc:2804] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:51.307930 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.303246 15232 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:05:51.306126 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 433164us
I20260709 13:05:51.304267 15230 raft_consensus.cc:1275] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:51.303081 15237 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:51.273671 22677 consensus_queue.cc:1048] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:05:51.330122 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:51.334981 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 479182us
I20260709 13:05:51.367488 15156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57498 (request call id 2850) took 1308 ms. Trace:
I20260709 13:05:51.367897 15156 rpcz_store.cc:276] 0709 13:05:50.059010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.152462 (+ 93452us) service_pool.cc:224] Handling call
0709 13:05:51.367314 (+1214852us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540017dc54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:51.367436 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435261696}
I20260709 13:05:51.369462 15154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2853) took 1309 ms. Trace:
I20260709 13:05:51.370702 22708 raft_consensus.cc:3060] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:51.369843 15154 rpcz_store.cc:276] 0709 13:05:50.059703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.163504 (+103801us) service_pool.cc:224] Handling call
0709 13:05:50.166059 (+  2555us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:50.173641 (+  7582us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:50.388123 (+214482us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.942896 (+554773us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.943076 (+   180us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:50.949448 (+  6372us) log.cc:844] Serialized 52 byte log entry
0709 13:05:50.949651 (+   203us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:50.949699 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.949785 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.369221 (+419436us) raft_consensus.cc:1685] finished
0709 13:05:51.369247 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.369407 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7109,"prepare.queue_time_us":260753,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":34307,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":21164,"spinlock_wait_cycles":5757312,"thread_start_us":697959,"threads_started":2,"wal-append.queue_time_us":691433}
I20260709 13:05:51.372341 22888 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:51.372714 22888 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
W20260709 13:05:51.374811 22925 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.446s	user 0.001s	sys 0.002s
I20260709 13:05:51.328876 22845 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:05:51.299386 22728 consensus_queue.cc:1059] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:05:51.299350 22872 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:05:51.350934 22389 consensus_queue.cc:1048] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:05:51.349010 22935 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 13:05:51.354379 22861 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.346269 22793 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:51.362007 22910 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:51.365599 15160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57498 (request call id 2857) took 1288 ms. Trace:
I20260709 13:05:51.325722 22457 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:51.379870 22938 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:51.379873 22871 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.444010 22781 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.443884 22871 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.444628 22781 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.446287 22871 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:51.381035 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 67088.7us
I20260709 13:05:51.335922 22691 raft_consensus.cc:697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:51.448185 22885 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.448212 22691 consensus_queue.cc:237] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.382815 22728 raft_consensus.cc:3055] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:51.448820 22885 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.449381 22728 raft_consensus.cc:740] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:51.450548 15842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1821) took 1055 ms. Trace:
I20260709 13:05:51.450472 22728 consensus_queue.cc:260] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.450996 15842 rpcz_store.cc:276] 0709 13:05:50.395331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.566089 (+170758us) service_pool.cc:224] Handling call
0709 13:05:50.566449 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.679913 (+113464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.450325 (+770412us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54008057d4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:51.450494 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259040000}
I20260709 13:05:51.451943 15840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59138 (request call id 1815) took 1536 ms. Trace:
W20260709 13:05:51.427289 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 154369us
I20260709 13:05:51.429690 15160 rpcz_store.cc:276] 0709 13:05:50.076888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.630732 (+553844us) service_pool.cc:224] Handling call
0709 13:05:50.631041 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.794708 (+163667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.794745 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.814877 (+ 20132us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.271036 (+456159us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:51.271091 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.290156 (+ 19065us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.365303 (+ 75147us) raft_consensus.cc:1685] finished
0709 13:05:51.365336 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.365526 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":28016,"thread_start_us":318853,"threads_started":1,"wal-append.queue_time_us":498568}
W20260709 13:05:51.385416 22931 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:51.446841 22781 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:51.443923 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 337358us
W20260709 13:05:51.381839 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 476297us
I20260709 13:05:51.452325 15840 rpcz_store.cc:276] 0709 13:05:49.915655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.915930 (+   275us) service_pool.cc:224] Handling call
0709 13:05:49.916264 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.060262 (+1143998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.451736 (+391474us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540036ced4. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:51.451893 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2629283200}
W20260709 13:05:51.425947 15202 leader_election.cc:336] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.459203 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.466249 15155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57510 (request call id 3285) took 1127 ms. Trace:
I20260709 13:05:51.475334 15155 rpcz_store.cc:276] 0709 13:05:50.338856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.470400 (+131544us) service_pool.cc:224] Handling call
0709 13:05:51.466047 (+995647us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54001d8654. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e82fa 00007f0d449a9b17 00007f0d3df246d4 00007f0d3df24662 00007f0d3df24591 00007f0d3df2450c 00007f0d3df237c2 00007f0d3dd1f48c 00007f0d3dd1e8c6 00007f0d3dd1e26f
0709 13:05:51.466178 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542772992}
I20260709 13:05:51.474436 15202 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:51.394023 22935 raft_consensus.cc:3055] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:51.385715 22865 raft_consensus.cc:3055] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:05:51.495222 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 112673us
I20260709 13:05:51.498910 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:51.494284 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 136176us
I20260709 13:05:51.367125 22677 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 13:05:51.385700 22862 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:51.385460 22748 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.511204 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:51.518921 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 44932.6us
I20260709 13:05:51.519130 15840 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:51.519606 15814 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:51.512135 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 53425.3us
I20260709 13:05:51.516623 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.521116 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.517581 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:51.522214 15160 raft_consensus.cc:2393] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:51.507872 22842 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.009s	sys 0.025s
I20260709 13:05:51.510619 22728 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:51.468922 22674 consensus_queue.cc:1059] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:05:51.507916 22865 raft_consensus.cc:740] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:51.494135 22828 consensus_queue.cc:1048] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 13:05:51.515184 22910 raft_consensus.cc:493] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.545187 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:51.505300 22935 raft_consensus.cc:740] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:51.532656 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:51.476452 22890 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:51.518748 22499 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:51.499294 22871 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:05:51.499022 22569 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:05:51.397703 22888 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.561682 15155 raft_consensus.cc:1275] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:05:51.550729 15233 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:51.556950 22890 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:51.554446 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:51.534624 22842 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.009s	sys 0.026s
W20260709 13:05:51.553658 15814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.571209 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:51.535537 22682 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:51.577018 15156 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:51.544829 22903 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:51.553594 22779 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:51.550047 22897 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:05:51.587143 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 161220us
I20260709 13:05:51.526090 22748 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.553956 22911 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.588698 15202 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 116 similar messages]
I20260709 13:05:51.604492 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:51.605787 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.606689 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:51.608183 15158 raft_consensus.cc:1275] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:51.611127 15155 raft_consensus.cc:1275] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:05:51.594544 15202 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.640107 15202 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:51.644596 15234 raft_consensus.cc:1240] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 13:05:51.646652 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 52142.4us
I20260709 13:05:51.588056 22779 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.585767 22674 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:51.562819 22499 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.573494 22910 raft_consensus.cc:515] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:51.690033 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 47708.3us
I20260709 13:05:51.638767 22890 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.052s
W20260709 13:05:51.691579 22944 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:05:51.727197 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 58613.8us
I20260709 13:05:51.643208 22682 raft_consensus.cc:697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:51.707582 22885 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.709215 22861 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.715521 22935 consensus_queue.cc:260] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.716753 22748 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.644408 22911 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.742633 22911 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.746146 22499 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:51.727219 22898 consensus_queue.cc:1048] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 13:05:51.724156 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:51.673990 22937 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:51.721585 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:51.669464 22779 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:51.666272 22752 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:51.644622 22897 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:51.645824 22872 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.080s
W20260709 13:05:51.738777 15053 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.771387 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:51.772818 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 214211us
I20260709 13:05:51.790534 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:51.804657 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.804527 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:51.805171 15847 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:51.763054 22898 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:51.773705 22935 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:51.790117 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 284046us
W20260709 13:05:51.821606 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.824415 15051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.824939 15202 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.825402 15202 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:51.814352 22675 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:51.816399 22898 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:51.826162 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 352096us
I20260709 13:05:51.812157 22674 consensus_queue.cc:1059] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
W20260709 13:05:51.843922 15202 leader_election.cc:336] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.840936 22752 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:51.871284 15847 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
W20260709 13:05:51.871627 15202 leader_election.cc:336] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.873905 15202 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:51.874738 15128 leader_election.cc:336] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:51.875476 15128 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:51.846016 22898 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:51.960034 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 83426.7us
W20260709 13:05:51.881292 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 62265.6us
I20260709 13:05:51.885195 22499 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:51.980369 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:51.983683 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 96832.9us
I20260709 13:05:51.981119 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:51.983893 15234 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:51.992679 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 121084us
I20260709 13:05:51.992142 15203 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:52.018628 15128 leader_election.cc:336] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.019363 15128 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:51.827473 22682 consensus_queue.cc:237] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:51.985060 22708 consensus_meta.cc:287] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.000s
W20260709 13:05:52.032828 13967 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 15113)
W20260709 13:05:52.026908 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.033593 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:52.034595 15128 leader_election.cc:336] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.034871 15128 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:52.036590 15234 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
W20260709 13:05:52.037595 15128 leader_election.cc:336] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.038111 15128 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:52.028569 15202 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:52.042784 15128 leader_election.cc:336] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.043515 15128 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.041970 22857 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:52.045367 22781 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:52.047219 15128 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.047711 15128 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.048602 22857 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.049412 22857 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.024655 22748 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:52.049996 15052 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 
I20260709 13:05:51.994410 22730 consensus_queue.cc:1048] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:05:52.035957 22499 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.042479 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.051185 22913 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.051684 22913 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:52.052461 15128 leader_election.cc:336] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.052958 15128 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.053297 22910 leader_election.cc:290] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.053876 22910 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:52.053982 22913 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.054927 22861 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:52.055521 22861 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.057225 22960 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:05:52.059063 22961 log.cc:927] Time spent T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
W20260709 13:05:52.060250 22967 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:05:52.061112 22973 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.061638 22963 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:05:52.061739 22973 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.066737 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:52.067377 15155 raft_consensus.cc:2376] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21. Re-sending same reply.
I20260709 13:05:51.990882 22969 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.068619 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44985.9us
I20260709 13:05:52.069414 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.070875 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.071017 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.071732 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:52.075241 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:52.061146 22959 log.cc:927] Time spent T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 13:05:52.044904 22793 consensus_meta.cc:287] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.006s
I20260709 13:05:52.086027 15847 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:52.042121 22939 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.064167 22973 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.064196 22781 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.042857 22828 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:52.087281 15202 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.043109 22884 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.023639 22520 consensus_queue.cc:1048] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:05:52.073588 22752 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.079725 22708 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:52.099218 15202 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:52.121181 15202 leader_election.cc:336] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.092969 22965 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.130422 22975 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.130890 22965 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.131094 22975 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.133354 22965 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.134550 22965 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.083160 22969 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.129048 22675 consensus_queue.cc:1059] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:52.116065 22939 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.127314 22924 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.946s	user 0.004s	sys 0.014s
I20260709 13:05:52.123394 15202 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:52.139411 22828 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:52.149893 15202 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.150432 15202 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:52.152454 22728 consensus_meta.cc:287] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.002s
W20260709 13:05:52.155952 15202 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.156478 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.138788 22884 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.138715 22708 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:52.167196 15202 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.167958 15202 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.118978 22781 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.161664 22752 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:52.187613 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 117104us
I20260709 13:05:52.170423 22828 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:52.146652 22965 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.193830 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:52.191525 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 200615us
I20260709 13:05:52.200753 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e4a6d3ac0a4d6baf95fb7d5c21b5f7" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:52.171579 22884 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:52.204000 15128 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.217011 15848 raft_consensus.cc:1275] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:52.246984 15128 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.239964 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:52.190322 22975 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.196779 22752 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.265318 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.196729 22861 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.262055 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:52.277455 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.268270 15158 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:52.278755 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.280144 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:52.282853 15158 raft_consensus.cc:2393] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 21: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:52.258401 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54418.5us
I20260709 13:05:52.205543 22897 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.277592 15054 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.282500 22975 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:52.286315 15128 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.295637 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:52.297282 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.266275 22982 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:52.303267 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:52.316483 15128 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.333676 15128 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:52.343735 15128 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.344481 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:52.335870 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 67198.3us
I20260709 13:05:52.352083 15054 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.301690 22897 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:52.354764 15128 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.255195 22708 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.321810 22714 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.301571 22981 raft_consensus.cc:493] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.279767 22779 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.358894 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:52.359315 15054 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:52.341267 22885 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:52.366788 22965 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:52.369558 22979 log.cc:927] Time spent T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:05:52.370133 22972 log.cc:927] Time spent T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:05:52.348875 22480 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:52.336551 22910 raft_consensus.cc:493] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.381748 22884 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.387669 22890 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.441078 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 81999.4us
I20260709 13:05:52.389047 22973 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.389204 22922 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:52.400137 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.398749 22885 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.443660 22688 raft_consensus.cc:493] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.387867 22675 consensus_queue.cc:1059] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 13:05:52.449972 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 257229us
I20260709 13:05:52.391129 22978 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:52.394061 22995 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:52.447235 22965 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:52.512759 15160 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:52.521811 15203 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
W20260709 13:05:52.523119 15813 leader_election.cc:336] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:52.525904 15128 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.452287 22781 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.423372 22898 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.469063 22728 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.469129 22922 raft_consensus.cc:493] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:52.541059 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.541626 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.542698 15233 raft_consensus.cc:2410] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:05:52.541987 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.453568 22969 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:52.503361 22714 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.445080 22910 raft_consensus.cc:515] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.394225 22981 raft_consensus.cc:515] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.542559 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.448920 22973 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.549696 15233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.549968 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.561689 15228 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 19.
I20260709 13:05:52.555361 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.542065 15231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:52.530330 15813 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.449524 22890 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.562903 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.567061 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.501909 22688 raft_consensus.cc:515] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.568778 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.589291 22695 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.594257 22688 leader_election.cc:290] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.595709 22779 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.564853 22714 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.597519 22991 log.cc:927] Time spent T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:05:52.564393 22748 raft_consensus.cc:2804] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:52.605836 15236 raft_consensus.cc:1275] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:52.549472 22728 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.597473 22885 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.577874 22675 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:52.628366 22714 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.630285 22898 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:52.632170 22973 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.613202 22965 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.576344 22691 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:05:52.555325 22922 raft_consensus.cc:515] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.611980 22890 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.620096 22708 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.625739 22981 leader_election.cc:290] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.626860 22865 consensus_queue.cc:260] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.633488 22965 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.640702 15128 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:52.653302 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 122971us
W20260709 13:05:52.653668 15128 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.617693 22978 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.662333 15813 leader_election.cc:336] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.652248 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:52.659996 22992 log.cc:927] Time spent T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:05:52.679987 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 112424us
W20260709 13:05:52.687170 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 336935us
I20260709 13:05:52.663416 22981 raft_consensus.cc:493] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.700183 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.661758 22748 raft_consensus.cc:697] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:52.700953 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.702189 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:52.704178 15236 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:52.706197 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.650533 22708 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.682274 22865 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:52.721719 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:52.722406 15163 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:52.710630 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.713337 22569 raft_consensus.cc:3055] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:52.714274 15845 raft_consensus.cc:1240] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:52.715517 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:52.705231 22981 raft_consensus.cc:515] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.703922 22695 consensus_queue.cc:1048] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:05:52.703091 22980 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:52.719640 23005 log.cc:927] Time spent T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:52.726347 22728 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:52.726943 22922 leader_election.cc:290] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.666896 22978 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.731464 22965 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.733259 22708 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.747529 15235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41710 (request call id 1777) took 1380 ms. Trace:
I20260709 13:05:52.855322 15235 rpcz_store.cc:276] 0709 13:05:51.367618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.428030 (+ 60412us) service_pool.cc:224] Handling call
0709 13:05:51.428562 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.670296 (+241734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.670326 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.670374 (+    48us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:51.670415 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.747288 (+1076873us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400176e54. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:52.747337 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.747469 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555360512}
I20260709 13:05:52.809736 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:52.811754 22939 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:52.818553 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.820029 15051 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.861124 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:52.820914 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 137471us
I20260709 13:05:52.821026 15229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43960 (request call id 2862) took 1817 ms. Trace:
I20260709 13:05:52.861959 15229 rpcz_store.cc:276] 0709 13:05:51.004075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.005295 (+  1220us) service_pool.cc:224] Handling call
0709 13:05:51.005571 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.214468 (+208897us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:51.527068 (+312600us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.072643 (+545575us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.206113 (+133470us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.206187 (+    74us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:52.206252 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.820801 (+614549us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400319054. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9e4e01 00007f0d3f9e383e 00007f0d449a8419 00007f0d3df1fbd4 00007f0d3df1fb62 00007f0d3df1fa91 00007f0d3df1fa0c 00007f0d3df1ecc2 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:52.820866 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.820973 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":307243,"prepare.queue_time_us":609066,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":2126,"raft.queue_time_us":59764,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":38099,"spinlock_wait_cycles":479813376,"thread_start_us":670841,"threads_started":3,"wal-append.queue_time_us":316481}
I20260709 13:05:52.835367 22954 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.837697 22966 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.841665 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:52.842257 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 136387us
W20260709 13:05:52.843158 15202 leader_election.cc:336] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.746608 22569 raft_consensus.cc:740] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:52.977571 15202 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
W20260709 13:05:52.978520 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 135412us
I20260709 13:05:52.978281 22569 consensus_queue.cc:260] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.979575 23008 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:52.979720 22569 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:52.849757 23007 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:05:52.799268 15201 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:52.981251 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 182060us
I20260709 13:05:52.982084 22828 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:52.854141 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:52.984184 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 130161us
I20260709 13:05:52.984288 22828 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.797721 22938 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.984938 22828 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.873008 22708 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:52.986354 22708 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:52.873441 22966 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.887476 15163 raft_consensus.cc:2468] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 22.
W20260709 13:05:52.888495 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 69969.2us
I20260709 13:05:52.879267 22815 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:52.989885 22815 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:52.990415 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:52.991039 15813 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:52.991979 22815 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.867952 22748 consensus_queue.cc:237] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:52.879560 23010 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:52.995250 23010 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.997565 23010 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:52.998477 15126 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:52.872936 22954 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.005205 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:53.005847 15848 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:05:53.006528 22954 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.914476 22730 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:53.007551 22730 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.009939 22730 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:52.943176 22888 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:52.947311 23006 log.cc:927] Time spent T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:05:52.934064 22913 raft_consensus.cc:3055] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:53.011386 22913 raft_consensus.cc:740] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:53.011533 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efef46e0b3c14af2a7179beba5c160b4" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.012352 22913 consensus_queue.cc:260] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:52.956777 22977 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.935209 23011 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:05:52.958254 22978 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.958818 22981 leader_election.cc:290] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:52.959344 22910 leader_election.cc:290] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:52.961369 22935 consensus_meta.cc:287] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 1.141s	user 0.013s	sys 0.000s
I20260709 13:05:52.966766 22872 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:53.012748 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.014819 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.030462 22978 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.031023 22978 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.032428 15126 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.033411 13967 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:53.033250 22913 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:53.033694 22872 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.035909 22872 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:52.985224 22938 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.039937 22938 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:52.869165 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 49203.5us
I20260709 13:05:52.893812 22520 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:53.002388 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.041358 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:53.041373 15845 raft_consensus.cc:2410] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:05:53.042603 22977 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.045370 22935 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:53.045930 22935 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.046396 22935 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:53.047858 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.048755 22977 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.049552 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.049742 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:52.988673 22966 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.050155 22978 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.028004 15054 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.053153 14973 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ff3342a5b2ad49e2b252d7cd47b9851d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d45101e0851484690b4f03a4dcdb63d is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:53.055821 22708 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:53.057051 22884 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.057119 22910 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.058007 22884 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.058923 22884 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.060554 22777 raft_consensus.cc:493] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.058948 15126 leader_election.cc:304] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.064714 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 144188us
I20260709 13:05:53.071261 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:53.080267 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.081029 15230 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 19.
I20260709 13:05:53.084290 15051 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:53.086410 15848 raft_consensus.cc:2468] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 23.
I20260709 13:05:53.093532 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:53.061587 23020 log.cc:927] Time spent T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.096s	user 0.003s	sys 0.003s
W20260709 13:05:53.095984 23023 log.cc:927] Time spent T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.123s	user 0.001s	sys 0.003s
W20260709 13:05:53.118799 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 138845us
W20260709 13:05:53.119685 15813 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.120184 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:53.123375 15813 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:53.124166 15813 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.124472 15813 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.072944 22499 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.094799 22480 raft_consensus.cc:2749] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:53.100495 22981 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:53.101701 22954 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:53.138774 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.129669 23010 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:53.130924 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:53.133473 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:53.127177 15127 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.152180 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:53.147559 15814 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:53.128226 15126 leader_election.cc:304] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.158577 15127 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:05:53.159943 15126 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.161252 22996 raft_consensus.cc:2804] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:53.161841 22996 raft_consensus.cc:697] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:53.162679 22996 consensus_queue.cc:237] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.165117 22781 raft_consensus.cc:2749] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:53.165797 22781 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:53.167018 22977 raft_consensus.cc:493] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.167565 22977 raft_consensus.cc:515] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.169844 22977 leader_election.cc:290] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.171373 15230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.173046 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.174862 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.175403 15154 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:53.177064 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.179533 15163 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:05:53.179957 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.181038 15154 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:53.185209 22977 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.185776 22977 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.187209 15230 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:53.188797 22966 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:53.189224 22977 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.192466 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.195372 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.196276 15160 raft_consensus.cc:2410] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:53.195492 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.197156 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.198271 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:53.160526 15813 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.198869 15159 raft_consensus.cc:2468] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:53.200074 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.200071 15813 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:53.063141 22777 raft_consensus.cc:515] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.206212 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.115643 22938 raft_consensus.cc:493] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.210139 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 56043.1us
W20260709 13:05:53.210084 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.216758 15054 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.218319 15054 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.211138 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 55387.1us
I20260709 13:05:53.118512 22885 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.219728 15054 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:53.149080 22981 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.148905 22954 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:53.215452 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 55086.7us
I20260709 13:05:53.125226 22730 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:05:53.234985 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:53.243870 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 83995.7us
I20260709 13:05:53.251147 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:53.217465 23004 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:53.227972 22978 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:53.263089 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf60e792da464094a6aa2bfbcc5d71f2" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.137550 22929 consensus_queue.cc:1048] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:05:53.219748 22854 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:53.276046 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.278230 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365e6a51b3034d0483fdd8fc4cd362ea" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:53.261606 15813 leader_election.cc:336] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.221241 22884 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.261650 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f228ea0b4cc6831089c6ddf8ec96" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.150265 22499 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.233418 22708 raft_consensus.cc:2804] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:53.234227 22981 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:53.246456 22954 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:53.233165 22977 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:53.300048 15813 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.258030 23002 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.299006 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 21.
I20260709 13:05:53.317386 15813 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:53.275331 22958 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:53.309146 15203 leader_election.cc:304] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:53.312000 15236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d62d65616394d059c479b07e82eccb9" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:53.322031 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 174596us
I20260709 13:05:53.231925 22938 raft_consensus.cc:515] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.328742 15841 raft_consensus.cc:1240] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:53.329293 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.330542 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.331851 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9063404f3d4db9b81ce66060eee5e7" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.333232 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.244431 22885 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.334429 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.336223 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.336819 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.339761 15840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.339905 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.340736 15845 raft_consensus.cc:2393] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:53.341228 15840 raft_consensus.cc:1240] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Rejecting Update request from peer 12daa67457954a0cab27e32bf85fa115 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:53.341833 15845 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:53.342216 22499 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.302749 23013 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:53.344698 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.348402 15845 raft_consensus.cc:2393] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 23: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:53.321558 22911 raft_consensus.cc:2749] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.312801 22708 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.324115 22958 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:53.354060 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 54040.9us
W20260709 13:05:53.358673 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:53.362109 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ae83bb85154bb08a2889f0f9f99cd8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.363816 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.365770 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.334453 23002 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.373864 23002 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.351680 22520 raft_consensus.cc:2804] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:53.296864 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.455659 15052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.459785 15126 leader_election.cc:304] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.472368 22854 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.391654 22777 leader_election.cc:290] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.389035 22885 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.399474 22935 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.393828 22938 leader_election.cc:290] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.360746 22958 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:53.551924 15155 raft_consensus.cc:1275] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:53.359014 22708 raft_consensus.cc:3060] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:53.557286 22978 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.558337 22978 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.559041 22977 raft_consensus.cc:3060] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:53.398798 22981 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.400556 23012 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.583201 23012 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.584295 15154 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:53.356282 23013 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.591763 22828 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:53.537205 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 77930us
I20260709 13:05:53.537524 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.538049 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84610e2d65945cab0886909d43d5e9e" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.538267 22854 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.539645 22480 raft_consensus.cc:2749] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:53.480335 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.563815 22913 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.567965 22865 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.543032 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.359193 23018 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:53.592213 22520 raft_consensus.cc:697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:53.592777 22981 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.620520 22913 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.628223 22913 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.633721 22865 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.634904 22854 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.636101 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.637915 15848 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:53.637001 15844 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:53.636507 22865 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.674765 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:53.681241 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 126050us
I20260709 13:05:53.686142 23013 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.644948 22978 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:53.699635 22569 consensus_meta.cc:287] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.018s
I20260709 13:05:53.707129 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.717090 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 410406us
I20260709 13:05:53.720048 22520 consensus_queue.cc:237] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.708551 15126 leader_election.cc:304] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.742750 22779 raft_consensus.cc:2749] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:53.743001 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 86457.4us
I20260709 13:05:53.744278 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:53.744853 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:53.745446 15126 leader_election.cc:304] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:53.745767 15160 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 19.
W20260709 13:05:53.748160 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 124182us
I20260709 13:05:53.756999 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.757869 15235 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 20.
I20260709 13:05:53.765779 23012 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.765558 22815 consensus_queue.cc:1048] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:53.766685 22910 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:53.766980 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.767410 22910 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.771380 22779 raft_consensus.cc:2749] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:53.774505 23013 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:53.784374 22978 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:53.776701 22935 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:53.798081 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.798820 15842 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:53.783416 23013 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:53.806959 15128 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.810539 23013 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:53.821949 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 60066us
I20260709 13:05:53.824579 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.822851 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:53.832811 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 87527.1us
I20260709 13:05:53.845698 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:53.846091 22935 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.847012 22935 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.847755 22910 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.848690 22969 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:53.849146 22969 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.849335 15235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:53.850107 15235 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:53.871443 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 64520.3us
W20260709 13:05:53.875414 22954 consensus_meta.cc:287] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.006s
I20260709 13:05:53.876049 22954 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:53.877578 23040 log.cc:927] Time spent T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:05:53.885900 22935 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:53.887313 15235 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:53.851183 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:53.891366 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57618.4us
I20260709 13:05:53.904312 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.905282 15843 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 20.
I20260709 13:05:53.906997 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.907670 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.855520 22969 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:53.916397 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:53.928247 15814 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:53.923424 15128 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:53.930114 15051 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:53.940361 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 237303us
I20260709 13:05:53.943789 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.949218 22978 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:53.949820 22978 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:53.948920 22958 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.950950 22978 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:53.954471 22958 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:53.955080 22854 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:53.911058 22708 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:53.959152 15201 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:53.958681 23043 log.cc:927] Time spent T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:05:53.978533 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:53.980367 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:53.981289 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60216us
I20260709 13:05:53.982571 22997 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:53.983062 22997 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.983448 22997 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:53.987232 15127 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:53.996066 22910 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:53.996484 22910 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:53.996886 22910 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:54.001286 22842 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:54.001825 22842 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.002246 22842 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:54.004984 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 69547.6us
I20260709 13:05:54.009603 22910 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:53.981381 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 89840us
I20260709 13:05:53.994774 22997 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:54.018989 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:54.018455 15052 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.021822 22935 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:54.023115 15810 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:54.023240 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 57318.8us
I20260709 13:05:54.027941 15052 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.029091 22935 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.031193 15052 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:54.035836 13967 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:54.036573 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.036710 15848 raft_consensus.cc:1275] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:05:54.033605 22520 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:54.039163 22520 raft_consensus.cc:493] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.039559 22520 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:54.040205 22708 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.041003 22480 consensus_queue.cc:1048] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:05:54.041976 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:54.046486 15234 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 20: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
I20260709 13:05:54.043134 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0c065a7fec4f5c88b4bc85b3508231" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.048199 22954 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.050220 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:54.050983 15237 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 20: Already voted for candidate 4b633b84351f40de9437e3d532fe6fa7 in this term.
W20260709 13:05:54.064320 22977 consensus_meta.cc:287] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.003s
I20260709 13:05:54.066054 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.068097 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.072041 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a84a565612402dbb559ec973692018" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.074410 15163 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:54.073704 15156 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:54.079578 22913 raft_consensus.cc:3060] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:54.085793 22910 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.092002 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7defae86b18344bdb8b04e0d7c8f1a8d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.091919 22969 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.127928 15126 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.074052 22977 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.101256 23008 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:05:54.102031 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f14609939d04f1e90837cb17fe19457" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.103420 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.106652 15158 raft_consensus.cc:1275] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:54.110244 22997 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Requested vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.116192 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
W20260709 13:05:54.117357 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 86371.5us
W20260709 13:05:54.118777 23045 log.cc:927] Time spent T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:05:54.096192 23013 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:54.120365 15054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.122241 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43562.5us
I20260709 13:05:54.100203 22842 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.145363 15229 raft_consensus.cc:2368] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19.
I20260709 13:05:54.120435 22708 raft_consensus.cc:493] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:54.137082 23008 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:54.143311 15126 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.145186 15161 raft_consensus.cc:2393] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 20: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:54.099067 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.153330 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.154253 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.156324 15155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.156625 15162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.157974 15158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.158690 15158 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:54.159576 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.152791 15163 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 18.
I20260709 13:05:54.164099 15839 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 18.
I20260709 13:05:54.121846 14978 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 10 to 18, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 18 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.166533 22480 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:54.170919 22981 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:54.171227 22708 raft_consensus.cc:515] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.171870 23008 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:54.171656 15126 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.173480 22708 leader_election.cc:290] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.176849 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.178524 15052 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.179174 22981 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:54.179497 22935 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:54.181557 22935 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:54.182247 22935 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:54.183898 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.185125 15054 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.186190 23013 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.186759 22865 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:54.187052 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.187444 22865 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.188330 22520 raft_consensus.cc:515] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.188740 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:54.188853 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:54.189531 22977 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:54.189947 22977 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.190310 22977 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:54.190569 22520 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.191071 15234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:54.191766 15234 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 22: Already voted for candidate 12daa67457954a0cab27e32bf85fa115 in this term.
I20260709 13:05:54.191823 15160 raft_consensus.cc:2393] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12daa67457954a0cab27e32bf85fa115 in current term 22: Already voted for candidate 3d45101e0851484690b4f03a4dcdb63d in this term.
I20260709 13:05:54.196813 15844 raft_consensus.cc:2368] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:54.197276 15228 raft_consensus.cc:2368] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:54.168769 15127 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:54.204344 14983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 371) took 10728 ms. Trace:
I20260709 13:05:54.216223 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:05:54.211248 22935 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.221627 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.204921 14983 rpcz_store.cc:276] 0709 13:05:43.475673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.475897 (+   224us) service_pool.cc:224] Handling call
0709 13:05:54.204108 (+10728211us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:47.889689 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:47.925620 (+ 35931us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:47.925680 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:05:47.925703 (+    23us) tablet.cc:662] Decoding operations
0709 13:05:47.929121 (+  3418us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:47.929235 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:47.929261 (+    26us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:05:47.931398 (+  2137us) tablet.cc:701] Row locks acquired
0709 13:05:47.931461 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:05:47.931688 (+   227us) write_op.cc:270] Start()
0709 13:05:47.931875 (+   187us) write_op.cc:276] Timestamp: P: 1783602347931608 usec, L: 0
0709 13:05:47.931901 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:05:47.944439 (+ 12538us) log.cc:844] Serialized 9831 byte log entry
0709 13:05:49.106925 (+1162486us) op_driver.cc:464] REPLICATION: finished
0709 13:05:49.833961 (+727036us) write_op.cc:301] APPLY: starting
0709 13:05:49.834044 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:50.461197 (+627153us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b1800086620. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3f7230ec 00007f0d3f725af7 00007f0d3f726fb0 00007f0d4343b1bb 00007f0d4343b532 00007f0d43482b93 00007f0d4332fde3 00007f0d4332f6ab 00007f0d43330e77
0709 13:05:50.940719 (+479522us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:50.940744 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:52.372603 (+1431859us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:52.433247 (+ 60644us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:05:52.445432 (+ 12185us) write_op.cc:312] APPLY: finished
0709 13:05:52.614121 (+168689us) log.cc:844] Serialized 220 byte log entry
0709 13:05:52.705443 (+ 91322us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:52.794382 (+ 88939us) write_op.cc:454] Released schema lock
0709 13:05:52.802338 (+  7956us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:52.888729 (+ 86391us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:52.891254 (+  2525us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":659566,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":709504,"apply.run_cpu_time_us":65130,"apply.run_wall_time_us":3036877,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108777,"num_ops":25,"prepare.queue_time_us":382603,"prepare.run_cpu_time_us":34553,"prepare.run_wall_time_us":410054,"raft.queue_time_us":411596,"raft.run_cpu_time_us":51214,"raft.run_wall_time_us":328908,"replication_time_us":1174849,"spinlock_wait_cycles":78722304,"thread_start_us":1253336,"threads_started":5,"wal-append.queue_time_us":658284,"wal-append.run_cpu_time_us":73790,"wal-append.run_wall_time_us":2590798}]]}
I20260709 13:05:54.233779 22978 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:54.274731 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 124929us
I20260709 13:05:54.275094 22978 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:54.275586 15813 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.276046 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.277845 22978 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.278462 22978 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.279006 22978 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.280964 22708 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.281582 22708 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:54.271577 15201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 56548.4us
W20260709 13:05:54.286705 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 108355us
I20260709 13:05:54.289103 22929 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:54.289325 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.289753 22929 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:54.290901 22929 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.226163 22935 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.229157 14978 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:54.294726 22935 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.296155 22977 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.298393 22729 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.303514 22729 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.303936 23010 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.305294 22913 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.300663 22865 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.290925 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.295369 14978 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 12 to 16, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 16 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:54.302062 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 59494.9us
W20260709 13:05:54.308516 15813 leader_election.cc:336] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.309787 15813 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.282137 22969 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.311928 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.314076 15839 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:54.316183 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 101283us
I20260709 13:05:54.320794 22842 leader_election.cc:290] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.322156 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.322768 15154 raft_consensus.cc:3060] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:54.325564 22865 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:54.326536 22826 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:54.326059 15813 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.331342 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.332264 22389 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:54.327951 22865 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.295413 22935 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:54.341135 15839 raft_consensus.cc:2468] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:05:54.346897 22569 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.349660 15156 raft_consensus.cc:1275] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:05:54.350340 22729 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:54.351306 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 60536.2us
I20260709 13:05:54.352350 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.354179 22935 raft_consensus.cc:3060] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:54.355098 15128 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.343616 22978 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:54.361191 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.363606 15814 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.366703 22520 raft_consensus.cc:2764] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:54.369861 22728 consensus_queue.cc:1048] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:54.371654 22569 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.372340 22569 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.373158 22978 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.373833 22978 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.354234 15052 leader_election.cc:304] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:54.380270 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.389562 22865 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.390154 22865 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.392402 22865 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.394232 22978 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.388650 15154 raft_consensus.cc:2468] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 21.
I20260709 13:05:54.316942 22977 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.398054 15127 leader_election.cc:400] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:54.398494 22954 raft_consensus.cc:2749] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:54.400676 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:54.401015 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 46909.3us
I20260709 13:05:54.374704 22569 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.374565 14978 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.410174 15127 leader_election.cc:403] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:05:54.409996 15201 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:54.411518 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.409592 15203 leader_election.cc:304] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.422143 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.424868 15839 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
W20260709 13:05:54.430564 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 86122us
I20260709 13:05:54.408803 22708 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.438925 14978 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 16 to 17, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 17 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.441790 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.443586 15814 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.449455 22954 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:54.432705 22911 raft_consensus.cc:2749] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:54.450168 22954 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.436332 22958 raft_consensus.cc:2804] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:54.453269 22958 raft_consensus.cc:697] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:54.454217 22958 consensus_queue.cc:237] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.433760 23013 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:54.465600 22954 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.436156 22569 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.464397 14978 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 17 to 22, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 22 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:54.489477 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.489298 22569 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:54.490551 14978 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 12 to 16, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 16 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.480419 22389 raft_consensus.cc:2804] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:54.492259 22389 raft_consensus.cc:697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:05:54.494053 22389 consensus_queue.cc:237] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.496007 22935 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:54.485172 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 96378.5us
I20260709 13:05:54.498172 22911 consensus_queue.cc:1048] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:54.499280 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.488690 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.501571 15161 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:54.502800 15126 leader_election.cc:400] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19.
I20260709 13:05:54.503800 15126 leader_election.cc:400] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 20 pre-election: Vote denied by peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) with higher term. Message: Invalid argument: T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d for earlier term 20. Current term is 21.
I20260709 13:05:54.504784 22997 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.500034 14978 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 16 to 17, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 17 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:54.506466 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.507668 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4be666e4a740bd853d8f2748ad6191" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:54.470336 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 47047.2us
I20260709 13:05:54.510462 15052 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.511927 15052 leader_election.cc:304] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.512781 22978 raft_consensus.cc:2749] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.491569 22569 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.509461 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150ed0096c74899b55a5128304e91f6" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.516961 15163 raft_consensus.cc:2393] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b633b84351f40de9437e3d532fe6fa7 in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:54.511981 22708 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:54.507565 14978 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:54.534518 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.549656 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.553293 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:54.554108 15161 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:54.555006 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.555893 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.557852 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.559973 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.561029 23024 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.561514 23024 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.563695 23024 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:54.566697 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.595598 15161 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:54.600666 15157 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:05:54.601840 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.599520 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.603046 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:54.603328 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:54.606417 22954 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:54.608335 22954 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.608752 22954 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:54.617082 15054 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:54.620381 22708 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:54.620966 22708 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 23 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
W20260709 13:05:54.623389 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.628322 22569 raft_consensus.cc:493] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.632633 22569 raft_consensus.cc:515] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:54.635375 22977 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:05:54.640950 22569 leader_election.cc:290] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:54.624665 22708 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:54.651232 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.658035 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.658771 15844 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:54.663440 15839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.664978 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.664983 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cd34bca3be499d87220d6e1b2a763f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.667675 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.668474 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:54.670104 15813 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.670581 15813 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:54.671703 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:54.672276 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.672935 15846 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 21.
I20260709 13:05:54.696384 15848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6a82b88a6f4f6093256a0d85708d37" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:54.711846 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.712371 15813 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.713263 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.715876 15842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.719977 15203 leader_election.cc:304] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.723130 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b358d067cea0428190a484601dea7c69" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.722453 15051 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.730803 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.731142 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4010b839395412ca61c77e701fcb965" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.732933 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:54.743695 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12ada8057b141518113efeb519c6514" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.719000 22389 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:54.746614 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 78787us
I20260709 13:05:54.749239 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.750327 23060 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.751767 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.754966 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.759588 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.752862 23012 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:54.739419 22569 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:54.778854 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45379.8us
W20260709 13:05:54.779702 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.757884 22913 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:54.739903 22935 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.786895 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.789884 15847 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:54.785234 15051 leader_election.cc:304] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.776520 22708 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.793517 15052 leader_election.cc:304] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.794782 22978 raft_consensus.cc:493] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:54.810837 22978 raft_consensus.cc:515] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.796447 23058 raft_consensus.cc:2749] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.796895 22935 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.806349 22977 raft_consensus.cc:2749] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:54.818547 15237 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:54.806488 22954 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.835059 22954 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.807744 15844 raft_consensus.cc:1240] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:54.835999 15846 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:54.797020 22865 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.829118 15052 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.833010 22935 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.848804 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
W20260709 13:05:54.850992 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 45425.4us
I20260709 13:05:54.851529 15159 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:54.853055 15232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:54.851519 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.853752 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:54.852733 15052 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.841938 23060 raft_consensus.cc:2749] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:54.859076 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.861506 22954 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.850718 22935 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:54.861128 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:54.860554 22978 leader_election.cc:290] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.868432 15052 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.873044 23072 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:54.872932 23052 consensus_queue.cc:1059] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:05:54.869719 23060 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:54.873873 23052 consensus_queue.cc:1243] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Peer 4b633b84351f40de9437e3d532fe6fa7 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:54.874492 23052 raft_consensus.cc:3055] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:54.874828 23052 raft_consensus.cc:740] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:05:54.875667 23052 consensus_queue.cc:260] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.865337 22935 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.877449 22935 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:54.876976 22910 consensus_queue.cc:1175] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" responder_term: 19 status { last_received { term: 17 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3d45101e0851484690b4f03a4dcdb63d for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:54.858533 22997 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:54.893378 22997 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.895859 22997 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:54.932682 15846 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 23.
I20260709 13:05:54.941977 15159 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 24.
I20260709 13:05:54.944156 15052 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:54.946347 15054 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:05:54.929229 23052 raft_consensus.cc:3060] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:54.947196 22978 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:54.952917 23069 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:54.971278 23069 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 24 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:54.972245 23069 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:54.988240 15052 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:54.989543 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:54.987002 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:54.992660 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:54.994815 15202 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:55.003522 15160 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:05:55.017594 15160 pending_rounds.cc:85] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Aborting all ops after (but not including) 13
I20260709 13:05:55.003453 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:54.992947 22978 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:55.006959 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.022626 15202 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:55.036309 13967 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:55.035734 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:55.037953 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de505aa1e895447486f53629c5269998" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:55.037240 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:55.043983 15845 raft_consensus.cc:2393] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d45101e0851484690b4f03a4dcdb63d in current term 21: Already voted for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in this term.
I20260709 13:05:55.033656 15160 pending_rounds.cc:107] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:05:55.044891 15843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:55.045607 15843 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:55.046368 15160 raft_consensus.cc:2889] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.995481 23072 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:55.001889 22935 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:55.066217 22842 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:55.068182 15157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:55.068887 15157 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:05:55.070626 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:55.029399 22958 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:55.077528 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:55.079630 15846 raft_consensus.cc:1275] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:55.098268 15127 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:55.099695 15052 leader_election.cc:304] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:55.113056 23069 raft_consensus.cc:2749] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:55.117285 15203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 75839.6us
I20260709 13:05:55.117129 22958 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:55.105481 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 82897.6us
I20260709 13:05:55.139988 15229 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:05:55.142069 22911 consensus_queue.cc:1048] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:55.142149 15228 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:55.151479 15843 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 24.
I20260709 13:05:55.155411 15156 raft_consensus.cc:1275] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:55.132723 23073 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:55.164683 22389 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:55.174808 15157 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:55.176712 15054 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:05:55.187325 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.176602 22997 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:55.219679 22708 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:55.220423 22708 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Becoming Leader. State: Replica: 12daa67457954a0cab27e32bf85fa115, State: Running, Role: LEADER
I20260709 13:05:55.221627 22708 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:55.193792 23060 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:05:55.264391 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.344797 23075 log.cc:927] Time spent T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:05:55.353978 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.356676 23076 consensus_queue.cc:1048] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.116s
W20260709 13:05:55.406334 23087 log.cc:927] Time spent T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:05:55.407037 22935 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 election: Requested vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:55.413327 15843 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:05:55.418459 15846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:05:55.419088 15846 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:55.437289 15159 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:05:55.446774 22935 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:55.462879 23072 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.479933 15845 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:05:55.480540 23072 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:55.482827 23072 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:55.466126 22913 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:05:55.483982 15159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:55.491904 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:55.503240 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 62039.9us
I20260709 13:05:55.501353 23069 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 13:05:55.507743 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.531028 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.531834 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.535692 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:55.536777 22935 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:55.539628 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.543409 15161 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:05:55.550612 15846 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12daa67457954a0cab27e32bf85fa115 in term 22.
I20260709 13:05:55.565536 22935 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.009s
W20260709 13:05:55.579528 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.600001 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:55.660099 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 60185.5us
I20260709 13:05:55.701537 15159 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:05:55.701932 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.710629 22935 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:05:55.723640 23072 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:55.739814 23072 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:55.730919 23069 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.742069 23069 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:55.744375 23069 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:55.747677 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:55.749046 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:55.749414 23072 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:55.756559 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:55.765626 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447fc877e0f944fbb59ddd85ad25d8a9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:55.780441 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301fa97ba426402092eaf85685303b33" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:55.787302 15052 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:55.789016 22913 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:55.788677 15845 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Refusing update from remote peer 12daa67457954a0cab27e32bf85fa115: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:55.790961 15051 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:55.794267 23069 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:55.805053 23072 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:55.805706 14981 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 15 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:55.810901 14981 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 16 to 17, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:55.813225 14981 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 11 to 15, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 15 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.822227 14981 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 11 to 14, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 14 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:55.827951 14981 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 16 to 17, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.833880 14981 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 16 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:55.835731 14981 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 16 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:55.831486 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.853633 14981 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.855576 14981 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 12, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 12 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:55.854405 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:55.858246 14981 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 12 to 14, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 14 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:55.883625 14978 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 359) took 37876 ms (client timeout 15000 ms). Trace:
I20260709 13:05:55.883754 14981 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 11 to 16, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 16 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:55.894237 14981 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:55.896307 14981 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 10 to 11, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 11 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:55.884191 14978 rpcz_store.cc:269] 0709 13:05:18.012832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.016938 (+  4106us) service_pool.cc:224] Handling call
0709 13:05:55.883496 (+37866558us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:54.870561 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:54.871260 (+   699us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:54.871315 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:05:54.871332 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:54.872370 (+  1038us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:54.872489 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:54.872515 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:05:54.881990 (+  9475us) tablet.cc:701] Row locks acquired
0709 13:05:54.882043 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:54.882248 (+   205us) write_op.cc:270] Start()
0709 13:05:54.882401 (+   153us) write_op.cc:276] Timestamp: P: 1783602354882173 usec, L: 0
0709 13:05:54.882436 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:05:54.883161 (+   725us) log.cc:844] Serialized 3648 byte log entry
0709 13:05:55.533127 (+649966us) op_driver.cc:464] REPLICATION: finished
0709 13:05:55.688964 (+155837us) write_op.cc:301] APPLY: starting
0709 13:05:55.689077 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:55.710354 (+ 21277us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:55.710471 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:55.722029 (+ 11558us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:55.722349 (+   320us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:05:55.722443 (+    94us) write_op.cc:312] APPLY: finished
0709 13:05:55.722796 (+   353us) log.cc:844] Serialized 89 byte log entry
0709 13:05:55.722976 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:55.723092 (+   116us) write_op.cc:454] Released schema lock
0709 13:05:55.723156 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:55.723604 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:55.723635 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":832877,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":148893,"apply.run_cpu_time_us":12746,"apply.run_wall_time_us":38078,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41801,"num_ops":9,"prepare.queue_time_us":97385,"prepare.run_cpu_time_us":81408,"prepare.run_wall_time_us":231625,"raft.queue_time_us":185981,"raft.run_cpu_time_us":69767,"raft.run_wall_time_us":126409,"replication_time_us":650493,"spinlock_wait_cycles":58240,"thread_start_us":481754,"threads_started":4,"wal-append.queue_time_us":292691}]]}
I20260709 13:05:56.032226 22935 raft_consensus.cc:493] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:56.033782 22935 raft_consensus.cc:515] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:56.038132 13967 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:56.038409 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:56.039659 22935 leader_election.cc:290] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.041504 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30c8fd833bca41f69ed253599a8889b9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:56.045163 15052 leader_election.cc:304] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:56.046509 22935 raft_consensus.cc:2749] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:56.095307 14981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 325) took 29431 ms (client timeout 15000 ms). Trace:
I20260709 13:05:56.104488 14977 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.113595 14977 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 18, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.126606 14977 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 11 to 16, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 16 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:56.095786 14981 rpcz_store.cc:269] 0709 13:05:26.664376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.664708 (+   332us) service_pool.cc:224] Handling call
0709 13:05:56.095215 (+29430507us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:55.967648 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:55.968156 (+   508us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:55.968185 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:05:55.968202 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:55.970355 (+  2153us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:55.970427 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:55.970459 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:05:55.971567 (+  1108us) tablet.cc:701] Row locks acquired
0709 13:05:55.971586 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:05:55.971726 (+   140us) write_op.cc:270] Start()
0709 13:05:55.971841 (+   115us) write_op.cc:276] Timestamp: P: 1783602355971685 usec, L: 0
0709 13:05:55.971862 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:05:55.972648 (+   786us) log.cc:844] Serialized 5192 byte log entry
0709 13:05:55.988270 (+ 15622us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b5400367fa8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d45da09e1 00007f0d45d9f64c 00007f0d4341b3bb 00007f0d4341d71a 00007f0d4341d6a8 00007f0d4341d628 00007f0d4341d5e3 00007f0d4341c7fc 00007f0d4584f626 00007f0d45849dd8 00007f0d3f9f0310
0709 13:05:55.988401 (+   131us) op_driver.cc:464] REPLICATION: finished
0709 13:05:55.990077 (+  1676us) write_op.cc:301] APPLY: starting
0709 13:05:55.990163 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:55.998587 (+  8424us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:55.998608 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:56.014275 (+ 15667us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:56.014533 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:05:56.014588 (+    55us) write_op.cc:312] APPLY: finished
0709 13:05:56.014902 (+   314us) log.cc:844] Serialized 121 byte log entry
0709 13:05:56.015034 (+   132us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.015137 (+   103us) write_op.cc:454] Released schema lock
0709 13:05:56.015173 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:56.022435 (+  7262us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.022468 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1553927,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":1426,"apply.run_cpu_time_us":16578,"apply.run_wall_time_us":32273,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54418,"num_ops":13,"prepare.queue_time_us":49615,"prepare.run_cpu_time_us":4194,"prepare.run_wall_time_us":5351,"replication_time_us":4219,"spinlock_wait_cycles":8903296,"thread_start_us":54949,"threads_started":1}]]}
W20260709 13:05:56.141645 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.136019 14977 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 14 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 18 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.152757 22935 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.153493 22935 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:56.152972 14977 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.155757 22935 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:56.157716 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.168951 22935 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.169557 22935 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:56.169919 14977 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 18, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.171934 22935 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:56.162276 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211046e319804173a9b8954112b8d044" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:56.176330 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.172797 14977 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.187052 14977 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 16, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 16 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.193738 14977 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 18, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 18 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.202153 14977 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 13 to 15, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 15 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.191887 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52fa48697b145c98311ef7704e35002" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:56.178478 15051 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:56.211513 23072 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:56.214840 14977 catalog_manager.cc:5697] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 17, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.219244 14977 catalog_manager.cc:5697] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 9 to 15, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 15 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.222747 15051 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:56.224128 23072 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:56.224108 14977 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 13 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 18 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:56.236114 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.243178 14977 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.241860 22913 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.245450 22913 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.241110 23072 raft_consensus.cc:493] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.247782 22913 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.247738 23072 raft_consensus.cc:515] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.250269 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:56.250612 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.256876 23072 leader_election.cc:290] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.259192 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:56.263301 14977 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.268816 14977 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 19, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.277869 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bf5d5f44ec498499e1cb1be55c2283" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:56.277989 15052 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:56.280257 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.280635 15052 leader_election.cc:304] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:56.289597 23072 raft_consensus.cc:2749] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:56.280716 22935 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:56.292373 14977 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 18, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:56.295019 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.322953 22935 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.323668 22935 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.331734 22935 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.332526 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.332763 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:56.349395 15054 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:56.350386 23072 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:56.355870 22935 raft_consensus.cc:493] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.357582 22935 raft_consensus.cc:515] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.365829 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:56.367432 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd68ef635784c42a7cc0ce9f2fae0f5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:56.369526 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.373641 15052 leader_election.cc:304] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:56.374540 23069 raft_consensus.cc:2749] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:56.381009 22935 leader_election.cc:290] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:56.520157 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.520833 14978 catalog_manager.cc:5697] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:56.535706 14977 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 266) took 19185 ms (client timeout 15000 ms). Trace:
I20260709 13:05:56.536360 14978 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 19, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.548096 14978 catalog_manager.cc:5697] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 22, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 22 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:56.536224 14977 rpcz_store.cc:269] 0709 13:05:37.349989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.350265 (+   276us) service_pool.cc:224] Handling call
0709 13:05:56.535597 (+19185332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:56.326127 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:56.326663 (+   536us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:56.326707 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:05:56.326732 (+    25us) tablet.cc:662] Decoding operations
0709 13:05:56.328173 (+  1441us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:56.328248 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:56.328278 (+    30us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:05:56.329610 (+  1332us) tablet.cc:701] Row locks acquired
0709 13:05:56.329631 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:05:56.329775 (+   144us) write_op.cc:270] Start()
0709 13:05:56.329898 (+   123us) write_op.cc:276] Timestamp: P: 1783602356329729 usec, L: 0
0709 13:05:56.329921 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:05:56.330617 (+   696us) log.cc:844] Serialized 6844 byte log entry
0709 13:05:56.333713 (+  3096us) op_driver.cc:464] REPLICATION: finished
0709 13:05:56.334009 (+   296us) write_op.cc:301] APPLY: starting
0709 13:05:56.334090 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:56.355824 (+ 21734us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:56.355905 (+    81us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:56.395701 (+ 39796us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:56.395998 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:05:56.396043 (+    45us) write_op.cc:312] APPLY: finished
0709 13:05:56.396402 (+   359us) log.cc:844] Serialized 156 byte log entry
0709 13:05:56.416828 (+ 20426us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.417093 (+   265us) write_op.cc:454] Released schema lock
0709 13:05:56.417153 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:56.417620 (+   467us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.417656 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":157391,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":87,"apply.run_cpu_time_us":21909,"apply.run_wall_time_us":83534,"cfile_cache_hit":68,"cfile_cache_hit_bytes":71162,"num_ops":17,"prepare.queue_time_us":6185,"prepare.run_cpu_time_us":4794,"prepare.run_wall_time_us":7287,"replication_time_us":3649}]]}
I20260709 13:05:56.589464 14978 catalog_manager.cc:5697] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.591610 14978 catalog_manager.cc:5697] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.593268 14978 catalog_manager.cc:5697] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 20 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.594992 14978 catalog_manager.cc:5697] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:56.606436 15202 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 232 similar messages]
I20260709 13:05:56.596285 14978 catalog_manager.cc:5697] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 17, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.622035 14978 catalog_manager.cc:5697] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 21, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.623718 14978 catalog_manager.cc:5697] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 20, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 20 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.625360 14978 catalog_manager.cc:5697] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.626837 14978 catalog_manager.cc:5697] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.627990 14978 catalog_manager.cc:5697] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.644434 14978 catalog_manager.cc:5697] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 15, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 15 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.646023 14978 catalog_manager.cc:5697] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.647228 14978 catalog_manager.cc:5697] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 17 to 18, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 18 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.648559 14978 catalog_manager.cc:5697] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 21 to 22, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 22 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.666148 14978 catalog_manager.cc:5697] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 20, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 20 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.667780 14978 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 16 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 18 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.669149 14978 catalog_manager.cc:5697] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.670615 14978 catalog_manager.cc:5697] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.671916 14978 catalog_manager.cc:5697] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 15 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.682320 22958 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.689157 22958 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:56.691272 22958 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:56.692448 14978 catalog_manager.cc:5697] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 12 to 18, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 18 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.693782 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.695855 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:56.694105 14978 catalog_manager.cc:5697] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 15 to 17, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:56.697744 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.701957 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:56.701598 14978 catalog_manager.cc:5697] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.703564 23084 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:56.704039 23084 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.704454 23084 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:56.703806 14978 catalog_manager.cc:5697] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 20, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 20 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.705766 14978 catalog_manager.cc:5697] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 19, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.707399 14978 catalog_manager.cc:5697] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 15 to 18, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 18 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.725224 14978 catalog_manager.cc:5697] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.726779 14978 catalog_manager.cc:5697] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 11 to 18, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 18 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.728487 14978 catalog_manager.cc:5697] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.730329 14978 catalog_manager.cc:5697] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 21, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 21 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.731734 14978 catalog_manager.cc:5697] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 21, leader changed from 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 21 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:56.749123 14978 catalog_manager.cc:5697] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:56.752493 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:56.797143 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 44593.2us
W20260709 13:05:56.792452 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:56.783349 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:56.819123 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:56.825608 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.796238 23084 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:56.863268 22935 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.863883 22935 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.866103 22935 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.882705 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:56.886399 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.890987 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afe5ac407a04216b625837a5bca525f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:56.899570 15054 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:56.921087 22935 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:56.936583 22935 raft_consensus.cc:493] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.937345 22935 raft_consensus.cc:515] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.942178 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
W20260709 13:05:56.905385 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:56.918349 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:56.944263 15163 raft_consensus.cc:3060] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:56.953348 15163 raft_consensus.cc:2468] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
I20260709 13:05:56.954653 15203 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:56.955610 23104 raft_consensus.cc:2804] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:56.956202 23104 raft_consensus.cc:697] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:56.957172 23104 consensus_queue.cc:237] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:56.919711 15202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 72146.5us
W20260709 13:05:56.976622 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 71289.7us
I20260709 13:05:56.978924 23072 raft_consensus.cc:493] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.979559 23072 raft_consensus.cc:515] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:56.980700 23084 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:56.981877 23072 leader_election.cc:290] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:56.985646 14981 catalog_manager.cc:5697] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 16 to 21, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 21 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:56.987443 14981 catalog_manager.cc:5697] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 19 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 21 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:56.988149 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 51896.3us
W20260709 13:05:56.989025 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:56.991423 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:56.997295 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:56.997743 14981 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 22, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 22 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
W20260709 13:05:57.050562 15128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 53317.9us
I20260709 13:05:57.030153 23072 raft_consensus.cc:493] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.052618 23072 raft_consensus.cc:515] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.055104 23072 leader_election.cc:290] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.033978 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07a1de4f2ab24624aa87e5619961710d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.058301 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.037206 22935 leader_election.cc:290] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:57.038621 13967 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:57.050927 14981 catalog_manager.cc:5697] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 19 to 23, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 23 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.061053 14981 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 11 to 16, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 16 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.062983 14981 catalog_manager.cc:5697] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 19, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 19 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
W20260709 13:05:57.067211 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:57.067685 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.006071 14978 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 267) took 2525 ms. Trace:
I20260709 13:05:57.075488 14978 rpcz_store.cc:276] 0709 13:05:54.481062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.938078 (+1457016us) service_pool.cc:224] Handling call
0709 13:05:56.018598 (+ 80520us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b5400367fa8. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d45bd6331 00007f0d45bd13fc 00007f0d457a31b4 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:57.005992 (+987394us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:56.787109 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:56.787638 (+   529us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:56.787681 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:56.787695 (+    14us) tablet.cc:662] Decoding operations
0709 13:05:56.790073 (+  2378us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:56.790130 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:56.790160 (+    30us) tablet.cc:685] Acquiring locks for 34 operations
0709 13:05:56.792502 (+  2342us) tablet.cc:701] Row locks acquired
0709 13:05:56.792520 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:05:56.792682 (+   162us) write_op.cc:270] Start()
0709 13:05:56.792799 (+   117us) write_op.cc:276] Timestamp: P: 1783602356792636 usec, L: 0
0709 13:05:56.792818 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:56.793497 (+   679us) log.cc:844] Serialized 13402 byte log entry
0709 13:05:56.796878 (+  3381us) op_driver.cc:464] REPLICATION: finished
0709 13:05:56.797370 (+   492us) write_op.cc:301] APPLY: starting
0709 13:05:56.797458 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:56.808196 (+ 10738us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b180040ab60. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d457bfde1 00007f0d457cc5b1 00007f0d3c8f7522 00007f0d3c8f97c1 00007f0d3c8f9779 00007f0d3c8f9709 00007f0d3c8f96d1 00007f0d3c8f89cd 000000000038f354 000000000038f199 00007f0d3c8da83a
0709 13:05:56.841912 (+ 33716us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:56.841941 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:56.928537 (+ 86596us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:56.928946 (+   409us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=68,key_file_lookups=68,delta_file_lookups=0,mrs_lookups=0
0709 13:05:56.928993 (+    47us) write_op.cc:312] APPLY: finished
0709 13:05:56.929518 (+   525us) log.cc:844] Serialized 292 byte log entry
0709 13:05:56.930464 (+   946us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.930731 (+   267us) write_op.cc:454] Released schema lock
0709 13:05:56.930831 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:56.951199 (+ 20368us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.951230 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":45740,"reported_tablets":45,"spinlock_wait_cycles":5684480,"child_traces":[["op",{"apply.queue_time_us":271,"apply.run_cpu_time_us":39950,"apply.run_wall_time_us":153769,"cfile_cache_hit":136,"cfile_cache_hit_bytes":142324,"num_ops":34,"prepare.queue_time_us":2518,"prepare.run_cpu_time_us":6694,"prepare.run_wall_time_us":17868,"replication_time_us":3891,"spinlock_wait_cycles":5942016}]]}
I20260709 13:05:57.064571 14981 catalog_manager.cc:5697] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 20, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 20 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.079792 14981 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 18 to 20, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 20 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.081334 14981 catalog_manager.cc:5697] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 18 to 20, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 20 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.082697 14981 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 22 to 26, leader changed from 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 26 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.097194 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50a21b24380415781c0e63c5d613aff" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.084141 14981 catalog_manager.cc:5697] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:57.105441 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 57456.2us
I20260709 13:05:57.105613 14981 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.118693 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bd17b479354abca0014ae059714e8f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:57.155174 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 80655.8us
I20260709 13:05:57.205822 14978 catalog_manager.cc:5697] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.207468 14978 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:57.208909 14978 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:57.210117 14978 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 20 to 24, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 12daa67457954a0cab27e32bf85fa115 (127.13.163.193). New cstate: current_term: 24 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:57.211707 14978 catalog_manager.cc:5697] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 18 to 19, leader changed from 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 19 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:57.217384 23069 raft_consensus.cc:493] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c28be6f86ea4fb6b79a47d3999f5a82)
I20260709 13:05:57.228905 23069 raft_consensus.cc:515] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.231343 23069 leader_election.cc:290] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.233681 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.233807 14978 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 13 to 22. New cstate: current_term: 22 leader_uuid: "12daa67457954a0cab27e32bf85fa115" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:57.234727 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.225951 15052 leader_election.cc:304] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:57.216759 15054 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 60808.5us
I20260709 13:05:57.241333 15054 leader_election.cc:304] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:57.242115 23069 raft_consensus.cc:2749] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:57.253609 15052 leader_election.cc:304] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.239259 14981 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59952 (request call id 372) took 1708 ms. Trace:
W20260709 13:05:57.237267 15202 leader_election.cc:336] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:57.259513 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30853833ff254a12bcc77a177a964bb9" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.253119 23069 raft_consensus.cc:2749] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:57.275605 15052 leader_election.cc:304] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.293823 23072 raft_consensus.cc:2749] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:57.296538 15845 consensus_queue.cc:237] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.258440 22935 raft_consensus.cc:2749] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:57.255349 14981 rpcz_store.cc:276] 0709 13:05:55.530615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.141804 (+611189us) service_pool.cc:224] Handling call
0709 13:05:57.239209 (+1097405us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:57.131676 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:57.132174 (+   498us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:57.132209 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:05:57.132225 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:57.133223 (+   998us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:57.133277 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:57.133302 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:05:57.134301 (+   999us) tablet.cc:701] Row locks acquired
0709 13:05:57.134327 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:05:57.134469 (+   142us) write_op.cc:270] Start()
0709 13:05:57.134582 (+   113us) write_op.cc:276] Timestamp: P: 1783602357134424 usec, L: 0
0709 13:05:57.134604 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:57.144353 (+  9749us) log.cc:844] Serialized 4837 byte log entry
0709 13:05:57.148341 (+  3988us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94d571 00007f0d3f9dfe28 00007f0d4341a581 00007f0d43419d58 00007f0d4341e9f1 00007f0d4341e9a9 00007f0d4341e939 00007f0d4341e901 00007f0d4341db2d 000000000038f354 000000000038f199
0709 13:05:57.153182 (+  4841us) op_driver.cc:464] REPLICATION: finished
0709 13:05:57.155536 (+  2354us) write_op.cc:301] APPLY: starting
0709 13:05:57.155618 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:57.161708 (+  6090us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:57.161736 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:57.172336 (+ 10600us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:57.172595 (+   259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:05:57.172651 (+    56us) write_op.cc:312] APPLY: finished
0709 13:05:57.172965 (+   314us) log.cc:844] Serialized 113 byte log entry
0709 13:05:57.183565 (+ 10600us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.183704 (+   139us) write_op.cc:454] Released schema lock
0709 13:05:57.183754 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:57.195352 (+ 11598us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.195382 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16334,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":2128,"apply.run_cpu_time_us":14131,"apply.run_wall_time_us":39778,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50232,"num_ops":12,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":5718,"prepare.run_wall_time_us":16822,"replication_time_us":18439,"spinlock_wait_cycles":20885760}]]}
W20260709 13:05:57.324821 15051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 65958.3us
W20260709 13:05:57.337167 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.344594 15161 raft_consensus.cc:1275] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:57.349172 23010 consensus_queue.cc:1048] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:57.352183 23069 raft_consensus.cc:493] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.352744 23069 raft_consensus.cc:515] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:57.356494 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
W20260709 13:05:57.358021 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 88284.3us
I20260709 13:05:57.363163 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1868a2755dea467c9c719191d9927234" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.368633 15051 leader_election.cc:304] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.369517 23111 raft_consensus.cc:2749] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:57.360896 23069 leader_election.cc:290] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:57.381850 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:57.378538 15052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 61234.3us
I20260709 13:05:57.395841 23069 raft_consensus.cc:493] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.396414 23069 raft_consensus.cc:515] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:57.401517 15229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.402307 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11eb27eddd0475ca037593dcdb66376" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.405719 15051 leader_election.cc:304] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:05:57.417279 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:57.418601 23010 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 8 to 13, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:05:57.424082 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 96733.2us
I20260709 13:05:57.430791 23111 raft_consensus.cc:2749] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:57.442188 15161 raft_consensus.cc:2955] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 8 to 13, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
I20260709 13:05:57.406648 23069 leader_election.cc:290] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:57.472703 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:57.478211 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:57.486629 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.489261 14971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff3342a5b2ad49e2b252d7cd47b9851d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:05:57.503171 15161 raft_consensus.cc:1275] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
W20260709 13:05:57.505848 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.509461 22958 consensus_queue.cc:1048] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:57.511935 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:57.515504 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:57.558010 14984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 356) took 120628 ms (client timeout 15000 ms). Trace:
W20260709 13:05:57.558595 14984 rpcz_store.cc:269] 0709 13:03:57.230403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.232145 (+  1742us) service_pool.cc:224] Handling call
0709 13:03:57.810299 (+578154us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400368154. stack: 00007f0d3c8c5343 00007f0d3c8c5240 00007f0d3c2541d4 00007f0d45f92304 00007f0d45f92299 00007f0d3f94e090 00007f0d3f9ecf3a 00007f0d457a31cc 00007f0d458a137e 00007f0d43d622a4 00007f0d43d62232 00007f0d43d62161 00007f0d43d620dc 00007f0d43d61392 00007f0d3dd1f48c 00007f0d3dd1e8c6
0709 13:05:57.556534 (+119746235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8983,"reported_tablets":45,"spinlock_wait_cycles":389365120}
I20260709 13:05:57.569723 14978 catalog_manager.cc:5697] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: config changed from index 8 to 13, VOTER 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195) evicted. New cstate: current_term: 19 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.589861 23072 raft_consensus.cc:493] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.590526 23072 raft_consensus.cc:515] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:57.594596 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.592711 23072 leader_election.cc:290] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:57.602118 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2d00b915f04988a8794ab25f22cbd5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.611858 15054 leader_election.cc:304] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.613106 23069 raft_consensus.cc:2749] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:57.614809 23072 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.615317 23072 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.619031 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.620693 23072 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.622030 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5f358b91ee427880ebecaca8118e4f" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.623724 15052 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.625270 23069 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:57.633544 23069 raft_consensus.cc:493] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.634193 23069 raft_consensus.cc:515] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:57.637565 23069 leader_election.cc:290] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:57.638154 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.639447 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af13e9e38f6a4785bef931855aafd50c" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.636425 14981 catalog_manager.cc:5697] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } }
I20260709 13:05:57.643340 23069 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:05:57.643939 23069 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.646281 23069 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.647719 15845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.651731 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca6bbc5df3084af4a938113f16eeedb5" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.653101 15051 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.654234 15054 leader_election.cc:304] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.655467 23072 raft_consensus.cc:2749] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:57.654304 23069 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:57.648072 14982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 264) took 67221 ms (client timeout 15000 ms). Trace:
W20260709 13:05:57.665269 14982 rpcz_store.cc:269] 0709 13:04:50.426902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.429226 (+  2324us) service_pool.cc:224] Handling call
0709 13:05:57.647857 (+67218631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3625,"reported_tablets":45}
I20260709 13:05:57.678478 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d with delete_type TABLET_DATA_TOMBSTONED (TS 4b633b84351f40de9437e3d532fe6fa7 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:41806
W20260709 13:05:57.731694 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.751250 23127 tablet_replica.cc:333] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:05:57.761497 23069 raft_consensus.cc:493] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.770788 23069 raft_consensus.cc:515] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.774331 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.774889 23069 leader_election.cc:290] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.774742 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.777164 15054 leader_election.cc:304] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.777554 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3342a5b2ad49e2b252d7cd47b9851d" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.774734 23127 raft_consensus.cc:2243] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:57.781172 23127 raft_consensus.cc:2272] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:57.782651 23069 raft_consensus.cc:2749] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:57.795300 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.819841 23127 ts_tablet_manager.cc:1916] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:57.848873 23069 raft_consensus.cc:493] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.849555 23069 raft_consensus.cc:515] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:57.854372 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:57.856245 23069 leader_election.cc:290] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:57.857142 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af12fc69de0b47b9adaf0267fb143776" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.859005 15051 leader_election.cc:304] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.860183 23069 raft_consensus.cc:2749] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:57.879031 14980 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 263) took 98113 ms (client timeout 15000 ms). Trace:
W20260709 13:05:57.892882 14980 rpcz_store.cc:269] 0709 13:04:19.766242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.773432 (+  7190us) service_pool.cc:224] Handling call
0709 13:05:57.878920 (+98105488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":384698,"reported_tablets":44}
W20260709 13:05:57.938422 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:57.939814 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:57.941504 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:57.954811 23127 ts_tablet_manager.cc:1929] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.12
I20260709 13:05:57.955544 23127 log.cc:1199] T ff3342a5b2ad49e2b252d7cd47b9851d P 4b633b84351f40de9437e3d532fe6fa7: Deleting WAL directory at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/wals/ff3342a5b2ad49e2b252d7cd47b9851d
I20260709 13:05:57.958750 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet ff3342a5b2ad49e2b252d7cd47b9851d (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:05:57.963397 23072 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.964322 23072 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:57.974053 23072 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:57.975167 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:57.975544 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d386fde0d10e4abf8b63290770e3ede0" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:57.985620 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d with delete_type TABLET_DATA_TOMBSTONED (Replica has no consensus available (current committed config index is 13)) from {username='slave'} at 127.0.0.1:41806
I20260709 13:05:57.990590 15051 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:57.991171 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet ff3342a5b2ad49e2b252d7cd47b9851d (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:05:57.992056 23069 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:57.999961 14986 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 322) took 109680 ms (client timeout 15000 ms). Trace:
W20260709 13:05:58.000955 14986 rpcz_store.cc:269] 0709 13:04:08.309862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.310138 (+   276us) service_pool.cc:224] Handling call
0709 13:05:57.990045 (+109679907us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":930325,"reported_tablets":45,"spinlock_wait_cycles":36096}
I20260709 13:05:58.003057 23072 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.008666 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.011219 23069 raft_consensus.cc:493] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.011897 23069 raft_consensus.cc:515] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:58.019376 23069 leader_election.cc:290] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:58.022440 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:58.008167 23072 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:58.045336 23072 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:58.045977 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe0a67b27bdc4c72abddb618266992a8" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:58.050334 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:58.058292 15054 leader_election.cc:304] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:58.060949 13967 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 15113)
I20260709 13:05:58.062891 23072 raft_consensus.cc:2749] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:58.067781 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:58.070649 14979 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59940 (request call id 360) took 13012 ms (client timeout 15000 ms). Trace:
W20260709 13:05:58.076416 14979 rpcz_store.cc:269] 0709 13:05:45.058234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.229821 (+2171587us) service_pool.cc:224] Handling call
0709 13:05:58.070427 (+10840606us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30292,"reported_tablets":45}
I20260709 13:05:58.094393 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df009dd21a4cb3867d92cb3a69c1e4" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:58.101804 23072 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.102430 23072 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:58.104607 23072 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:58.120692 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:58.120772 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:58.122730 13967 tablet_replica.cc:333] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.123605 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06f7ae7c1d614611bb074a65fcc9a959" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:58.124478 13967 raft_consensus.cc:2243] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.125582 13967 pending_rounds.cc:70] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.125871 13967 pending_rounds.cc:77] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 16 index: 8 } timestamp: 7305635170156564480 op_type: NO_OP noop_request { }
I20260709 13:05:58.126288 13967 raft_consensus.cc:2889] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.126713 13967 raft_consensus.cc:2272] T d386fde0d10e4abf8b63290770e3ede0 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.128254 15052 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:58.130374 13967 tablet_replica.cc:333] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.134176 13967 raft_consensus.cc:2243] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.134965 13967 pending_rounds.cc:70] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Trying to abort 2 pending ops.
I20260709 13:05:58.141561 23129 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:58.143252 15052 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:58.141044 13967 pending_rounds.cc:77] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305635163279491072 op_type: NO_OP noop_request { }
I20260709 13:05:58.144181 13967 raft_consensus.cc:2889] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.146346 13967 pending_rounds.cc:77] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305635124445413376 op_type: NO_OP noop_request { }
I20260709 13:05:58.146751 13967 raft_consensus.cc:2889] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.147189 13967 raft_consensus.cc:2272] T 7defae86b18344bdb8b04e0d7c8f1a8d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.147116 23128 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:58.162111 14985 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50368 (request call id 265) took 44597 ms (client timeout 15000 ms). Trace:
W20260709 13:05:58.162999 14985 rpcz_store.cc:269] 0709 13:05:13.564372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.621073 (+ 56701us) service_pool.cc:224] Handling call
0709 13:05:58.161914 (+44540841us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18254,"reported_tablets":45}
I20260709 13:05:58.175221 13967 tablet_replica.cc:333] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
W20260709 13:05:58.185209 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.189992 13967 raft_consensus.cc:2243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.190754 13967 pending_rounds.cc:70] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.191107 13967 pending_rounds.cc:77] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305635197459943424 op_type: NO_OP noop_request { }
I20260709 13:05:58.191534 13967 raft_consensus.cc:2889] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.197211 13967 raft_consensus.cc:2272] T 60df009dd21a4cb3867d92cb3a69c1e4 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.200635 13967 tablet_replica.cc:333] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.202206 13967 raft_consensus.cc:2243] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.205797 13967 pending_rounds.cc:70] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.206161 13967 pending_rounds.cc:77] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 15 index: 12 } timestamp: 7305635170087464960 op_type: NO_OP noop_request { }
I20260709 13:05:58.206578 13967 raft_consensus.cc:2889] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.207000 13967 raft_consensus.cc:2272] T 30853833ff254a12bcc77a177a964bb9 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.215703 13967 tablet_replica.cc:333] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.217180 13967 raft_consensus.cc:2243] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.217947 13967 raft_consensus.cc:2272] T 0a5f358b91ee427880ebecaca8118e4f P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.220995 13967 tablet_replica.cc:333] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.232775 23129 raft_consensus.cc:493] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.234369 23129 raft_consensus.cc:515] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:58.235042 13967 raft_consensus.cc:2243] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.235956 13967 raft_consensus.cc:2272] T af12fc69de0b47b9adaf0267fb143776 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.236958 23129 leader_election.cc:290] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:58.238674 15161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:58.248481 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb265c17a8bf4acfb7e480e4cf91d39b" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:58.251384 13967 tablet_replica.cc:333] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.252635 13967 raft_consensus.cc:2243] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.253494 13967 pending_rounds.cc:70] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 3 pending ops.
I20260709 13:05:58.260985 13967 pending_rounds.cc:77] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305635216084971520 op_type: NO_OP noop_request { }
I20260709 13:05:58.261544 13967 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.261893 13967 pending_rounds.cc:77] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305635167379902464 op_type: NO_OP noop_request { }
I20260709 13:05:58.262274 13967 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.262610 13967 pending_rounds.cc:77] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305635124566695936 op_type: NO_OP noop_request { }
I20260709 13:05:58.262959 13967 raft_consensus.cc:2889] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.263449 13967 raft_consensus.cc:2272] T 08bf5d5f44ec498499e1cb1be55c2283 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.259807 15051 leader_election.cc:304] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:58.266852 23129 raft_consensus.cc:2749] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:58.268692 15218 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3342a5b2ad49e2b252d7cd47b9851d with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:41806
I20260709 13:05:58.273000 13967 tablet_replica.cc:333] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.273670 14970 catalog_manager.cc:5028] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): tablet ff3342a5b2ad49e2b252d7cd47b9851d (table test-workload [id=aeaf2fa9dd1b49df8d9ce8b406932c01]) successfully deleted
I20260709 13:05:58.274135 13967 raft_consensus.cc:2243] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.260165 14983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59958 (request call id 326) took 7060 ms. Trace:
I20260709 13:05:58.277266 13967 raft_consensus.cc:2272] T d4010b839395412ca61c77e701fcb965 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.276961 14983 rpcz_store.cc:276] 0709 13:05:51.199963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.240046 (+3040083us) service_pool.cc:224] Handling call
0709 13:05:58.260074 (+4020028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18323,"reported_tablets":45}
W20260709 13:05:58.278043 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.280714 13967 tablet_replica.cc:333] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.289731 13967 raft_consensus.cc:2243] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:58.291978 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.297600 13967 raft_consensus.cc:2272] T de505aa1e895447486f53629c5269998 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.302161 23129 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.302793 23129 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:58.305346 23129 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:58.305928 13967 tablet_replica.cc:333] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.306188 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:58.306962 13967 raft_consensus.cc:2243] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.313491 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09eea6ae09f04b0b9937f09218fcb3b1" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:58.318401 13967 raft_consensus.cc:2272] T 9d8fe0d92f6f47dfb304a24bf915e2db P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.320441 15052 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:05:58.321789 23128 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:58.327509 13967 tablet_replica.cc:333] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.328666 13967 raft_consensus.cc:2243] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.329389 13967 raft_consensus.cc:2272] T 6b6a82b88a6f4f6093256a0d85708d37 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.335044 13967 tablet_replica.cc:333] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.339890 13967 raft_consensus.cc:2243] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.342833 13967 raft_consensus.cc:2272] T f52fa48697b145c98311ef7704e35002 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.381651 13967 tablet_replica.cc:333] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.382572 13967 raft_consensus.cc:2243] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.383390 13967 raft_consensus.cc:2272] T 0afe5ac407a04216b625837a5bca525f P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.386634 13967 tablet_replica.cc:333] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.387753 13967 raft_consensus.cc:2243] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.388893 13967 raft_consensus.cc:2272] T 808dd81f1b604329848aa94e3ecedc36 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.419474 13967 tablet_replica.cc:333] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.423134 13967 raft_consensus.cc:2243] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.425037 13967 pending_rounds.cc:70] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 2 pending ops.
I20260709 13:05:58.428121 13967 pending_rounds.cc:77] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305635176061702144 op_type: NO_OP noop_request { }
I20260709 13:05:58.428617 13967 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.429035 13967 pending_rounds.cc:77] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305635142179819520 op_type: NO_OP noop_request { }
I20260709 13:05:58.430814 13967 raft_consensus.cc:2889] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.431293 13967 raft_consensus.cc:2272] T 301fa97ba426402092eaf85685303b33 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.436677 13967 tablet_replica.cc:333] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.443190 13967 raft_consensus.cc:2243] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.457907 13967 raft_consensus.cc:2272] T 9a20274fd45144db87032cdf948f324d P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.467993 13967 tablet_replica.cc:333] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.469128 13967 raft_consensus.cc:2243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.469933 13967 pending_rounds.cc:70] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.474128 13967 pending_rounds.cc:77] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 20 index: 14 } timestamp: 7305635173818937344 op_type: NO_OP noop_request { }
I20260709 13:05:58.475260 13967 raft_consensus.cc:2889] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.485153 13967 raft_consensus.cc:2272] T 09eea6ae09f04b0b9937f09218fcb3b1 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.500049 13967 tablet_replica.cc:333] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.512871 13967 raft_consensus.cc:2243] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.513878 13967 raft_consensus.cc:2272] T 73ae83bb85154bb08a2889f0f9f99cd8 P 12daa67457954a0cab27e32bf85fa115 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.514647 23128 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.515265 23128 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:58.517407 13967 tablet_replica.cc:333] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.518546 13967 raft_consensus.cc:2243] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.519199 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:58.519724 13967 raft_consensus.cc:2272] T 41c6ed4c81c041d18f143b6cd51bea35 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.529024 23128 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:58.531262 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e777c555e1b4153913b5e6459164454" candidate_uuid: "12daa67457954a0cab27e32bf85fa115" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:58.533465 15054 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12daa67457954a0cab27e32bf85fa115; no voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d
W20260709 13:05:58.547719 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.546408 23128 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:58.551456 13967 tablet_replica.cc:333] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.552595 13967 raft_consensus.cc:2243] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.558787 13967 raft_consensus.cc:2272] T 447fc877e0f944fbb59ddd85ad25d8a9 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.566344 13967 tablet_replica.cc:333] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.577622 13967 raft_consensus.cc:2243] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.579035 13967 raft_consensus.cc:2272] T eaa49b1ae77944d7b6866e819d63d466 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.596817 13967 tablet_replica.cc:333] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.597821 13967 raft_consensus.cc:2243] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.598479 13967 raft_consensus.cc:2272] T cf9063404f3d4db9b81ce66060eee5e7 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.609997 13967 tablet_replica.cc:333] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.618052 13967 raft_consensus.cc:2243] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.618824 13967 raft_consensus.cc:2272] T af13e9e38f6a4785bef931855aafd50c P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.635317 13967 tablet_replica.cc:333] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.636279 13967 raft_consensus.cc:2243] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.636909 13967 raft_consensus.cc:2272] T 5e777c555e1b4153913b5e6459164454 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.677868 13967 tablet_replica.cc:333] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
W20260709 13:05:58.678049 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.698053 13967 raft_consensus.cc:2243] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.698715 13967 raft_consensus.cc:2272] T 07a1de4f2ab24624aa87e5619961710d P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.744799 13967 tablet_replica.cc:333] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.745761 13967 raft_consensus.cc:2243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.746331 13967 raft_consensus.cc:2272] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.769979 13967 tablet_replica.cc:333] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.770957 13967 raft_consensus.cc:2243] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.785542 13967 raft_consensus.cc:2272] T 14bd17b479354abca0014ae059714e8f P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:58.790025 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.794713 13967 tablet_replica.cc:333] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.795719 13967 raft_consensus.cc:2243] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.797004 13967 raft_consensus.cc:2272] T 661df855a98f493ca654519f6be2164f P 12daa67457954a0cab27e32bf85fa115 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:58.814358 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.811975 13967 tablet_replica.cc:333] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.816326 13967 raft_consensus.cc:2243] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.828297 13967 raft_consensus.cc:2272] T 15cd34bca3be499d87220d6e1b2a763f P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.848165 13967 tablet_replica.cc:333] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.849155 13967 raft_consensus.cc:2243] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:58.849875 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:58.850011 13967 raft_consensus.cc:2272] T 211046e319804173a9b8954112b8d044 P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.854914 13967 tablet_replica.cc:333] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.867657 13967 raft_consensus.cc:2243] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.868325 13967 pending_rounds.cc:70] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.868589 13967 pending_rounds.cc:77] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305635214777610240 op_type: NO_OP noop_request { }
I20260709 13:05:58.868992 13967 raft_consensus.cc:2889] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.869349 13967 raft_consensus.cc:2272] T 1868a2755dea467c9c719191d9927234 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.876538 13967 tablet_replica.cc:333] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.877521 13967 raft_consensus.cc:2243] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:58.879247 13967 raft_consensus.cc:2272] T 38a7fd74046f492f8ef8366fbc675ee0 P 12daa67457954a0cab27e32bf85fa115 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.896222 13967 tablet_replica.cc:333] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.897152 13967 raft_consensus.cc:2243] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.897893 13967 raft_consensus.cc:2272] T fe0a67b27bdc4c72abddb618266992a8 P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.905663 13967 tablet_replica.cc:333] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.906492 13967 raft_consensus.cc:2243] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.907051 13967 pending_rounds.cc:70] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.907363 13967 pending_rounds.cc:77] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305635208794079232 op_type: NO_OP noop_request { }
I20260709 13:05:58.907714 13967 raft_consensus.cc:2889] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.908049 13967 raft_consensus.cc:2272] T ee2d00b915f04988a8794ab25f22cbd5 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.915725 13967 tablet_replica.cc:333] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.916533 13967 raft_consensus.cc:2243] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.917182 13967 raft_consensus.cc:2272] T eb265c17a8bf4acfb7e480e4cf91d39b P 12daa67457954a0cab27e32bf85fa115 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.920004 13967 tablet_replica.cc:333] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.920827 13967 raft_consensus.cc:2243] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.921356 13967 pending_rounds.cc:70] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.921593 13967 pending_rounds.cc:77] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 16 index: 13 } timestamp: 7305635175820275712 op_type: NO_OP noop_request { }
I20260709 13:05:58.921960 13967 raft_consensus.cc:2889] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.922314 13967 raft_consensus.cc:2272] T ca6bbc5df3084af4a938113f16eeedb5 P 12daa67457954a0cab27e32bf85fa115 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.934492 13967 tablet_replica.cc:333] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.935371 13967 raft_consensus.cc:2243] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.936093 13967 raft_consensus.cc:2272] T ff3342a5b2ad49e2b252d7cd47b9851d P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.945132 13967 tablet_replica.cc:333] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.945935 13967 raft_consensus.cc:2243] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.946518 13967 pending_rounds.cc:70] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 2 pending ops.
I20260709 13:05:58.946761 13967 pending_rounds.cc:77] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 16 index: 14 } timestamp: 7305635169454821376 op_type: NO_OP noop_request { }
I20260709 13:05:58.947077 13967 raft_consensus.cc:2889] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.947362 13967 pending_rounds.cc:77] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 15 index: 13 } timestamp: 7305635142284611584 op_type: NO_OP noop_request { }
I20260709 13:05:58.947683 13967 raft_consensus.cc:2889] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.948011 13967 raft_consensus.cc:2272] T a12ada8057b141518113efeb519c6514 P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.950814 13967 tablet_replica.cc:333] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.951551 13967 raft_consensus.cc:2243] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.960553 13967 pending_rounds.cc:70] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.960857 13967 pending_rounds.cc:77] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 16 index: 12 } timestamp: 7305635189538385920 op_type: NO_OP noop_request { }
I20260709 13:05:58.961336 13967 raft_consensus.cc:2889] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.961687 13967 raft_consensus.cc:2272] T d50a21b24380415781c0e63c5d613aff P 12daa67457954a0cab27e32bf85fa115 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.964531 13967 tablet_replica.cc:333] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.965279 13967 raft_consensus.cc:2243] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.974205 13967 pending_rounds.cc:70] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Trying to abort 1 pending ops.
I20260709 13:05:58.974470 13967 pending_rounds.cc:77] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305635205773447168 op_type: NO_OP noop_request { }
I20260709 13:05:58.981194 13967 raft_consensus.cc:2889] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:58.981576 13967 raft_consensus.cc:2272] T 06f7ae7c1d614611bb074a65fcc9a959 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.984454 13967 tablet_replica.cc:333] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.985199 13967 raft_consensus.cc:2243] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.985849 13967 raft_consensus.cc:2272] T 30c8fd833bca41f69ed253599a8889b9 P 12daa67457954a0cab27e32bf85fa115 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:58.998162 13967 tablet_replica.cc:333] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:58.999068 13967 raft_consensus.cc:2243] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:58.999907 13967 raft_consensus.cc:2272] T b358d067cea0428190a484601dea7c69 P 12daa67457954a0cab27e32bf85fa115 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:59.017244 13967 tablet_replica.cc:333] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:59.017903 13967 raft_consensus.cc:2243] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:59.018909 13967 raft_consensus.cc:2272] T 16dab822009a4f00a0a480a48136ae25 P 12daa67457954a0cab27e32bf85fa115 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:59.021809 13967 tablet_replica.cc:333] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:59.022508 13967 raft_consensus.cc:2243] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:59.023069 13967 raft_consensus.cc:2272] T f11eb27eddd0475ca037593dcdb66376 P 12daa67457954a0cab27e32bf85fa115 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:59.024863 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:59.038076 13967 tablet_replica.cc:333] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115: stopping tablet replica
I20260709 13:05:59.038955 13967 raft_consensus.cc:2243] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:59.040095 13967 raft_consensus.cc:2272] T 740a0a73cde54c9abf2460f0bbdcf02d P 12daa67457954a0cab27e32bf85fa115 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:59.072985 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.112172 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:59.139601 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.253883 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.372442 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.401034 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:59.401718 23121 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.402354 23121 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:59.406883 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:59.407285 23121 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:05:59.451802 15128 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.452229 15128 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:59.454190 23121 raft_consensus.cc:2749] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:59.459367 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:59.459901 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.524664 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:59.557143 23121 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.557868 23121 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.563122 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:59.565052 23121 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:59.567780 15128 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.568327 15128 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:59.569149 23121 raft_consensus.cc:2749] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:59.659909 23107 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.660529 23107 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:59.666718 23107 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:59.680698 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:59.686167 23084 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.686759 23084 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:05:59.693450 15813 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.703507 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:59.709345 23112 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:59.729722 23084 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:59.732713 15847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:05:59.745122 15202 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.746997 15201 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:05:59.748262 23084 raft_consensus.cc:2749] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:59.754014 23130 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.754666 23130 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.757146 23130 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:05:59.762929 15128 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.768812 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:59.770944 15126 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:59.772428 23121 raft_consensus.cc:2749] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:59.777565 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:59.804747 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:59.805677 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:59.824615 23084 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.843053 22958 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.843880 22958 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:59.846107 22958 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:05:59.847187 23084 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.869166 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:05:59.870016 15844 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 18.
I20260709 13:05:59.873378 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:59.874327 15154 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 20.
I20260709 13:05:59.876073 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:59.877669 22958 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:59.878187 22958 raft_consensus.cc:493] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.878667 22958 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:59.894820 23010 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.895560 23010 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:05:59.897708 23010 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:59.904546 23138 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.905398 23138 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.919323 23084 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:59.921942 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:05:59.933319 15154 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 16.
I20260709 13:05:59.934907 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:05:59.941377 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.946736 15813 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.948765 23120 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:59.949292 15202 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.949533 23120 raft_consensus.cc:493] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
W20260709 13:05:59.950204 15202 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.939437 23138 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:59.951977 23120 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.962975 23121 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:05:59.963536 23121 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.959019 22958 raft_consensus.cc:515] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:05:59.965659 23121 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:05:59.967374 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:05:59.968705 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a20274fd45144db87032cdf948f324d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:05:59.969971 15154 raft_consensus.cc:3060] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:59.968959 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:59.970808 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:59.971282 15202 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.967355 22958 leader_election.cc:290] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:05:59.980150 15154 raft_consensus.cc:2468] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 21.
W20260709 13:05:59.980430 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:59.981457 15203 leader_election.cc:304] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:05:59.982432 22958 raft_consensus.cc:2804] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:59.983212 22958 raft_consensus.cc:697] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:05:59.984200 22958 consensus_queue.cc:237] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:05:59.985296 15128 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:05:59.986680 15128 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:05:59.988423 23121 raft_consensus.cc:2749] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:05:59.996989 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 284447us
I20260709 13:05:59.999089 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.001452 23084 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:00.001906 23084 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.002302 23084 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:00.011281 23137 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.011937 23137 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.013975 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:06:00.014436 23137 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.015930 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:06:00.016975 15154 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 26.
W20260709 13:06:00.017915 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.018905 15202 leader_election.cc:336] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.022684 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:06:00.023430 15160 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 22.
I20260709 13:06:00.024780 23137 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:06:00.025400 23137 raft_consensus.cc:493] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.025904 23137 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:06:00.031822 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:00.042863 23084 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:06:00.045584 15202 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.042889 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.036998 23120 raft_consensus.cc:515] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.049726 23120 leader_election.cc:290] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.051913 23140 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.052520 23140 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.053189 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c6ed4c81c041d18f143b6cd51bea35" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:06:00.053887 15160 raft_consensus.cc:3060] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.048043 23131 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:00.054802 23131 raft_consensus.cc:493] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.055238 23131 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:00.055953 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:06:00.056639 15228 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 22.
I20260709 13:06:00.057958 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:06:00.058784 23136 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:00.059154 23136 raft_consensus.cc:493] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.059499 23136 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:06:00.066445 15202 leader_election.cc:336] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.067338 23136 raft_consensus.cc:515] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.068940 23140 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:00.089200 15814 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:38049, user_credentials={real_user=slave}} blocked reactor thread for 46390.5us
I20260709 13:06:00.091202 14983 catalog_manager.cc:5697] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 20 to 21, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 21 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.101190 23136 leader_election.cc:290] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:06:00.102957 23137 raft_consensus.cc:515] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.088466 15160 raft_consensus.cc:2468] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 17.
I20260709 13:06:00.116725 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7fd74046f492f8ef8366fbc675ee0" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:06:00.117312 15203 leader_election.cc:304] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.106396 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa49b1ae77944d7b6866e819d63d466" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:06:00.118088 15237 raft_consensus.cc:3060] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:00.105805 23084 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:06:00.111079 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "808dd81f1b604329848aa94e3ecedc36" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:06:00.118979 23084 raft_consensus.cc:2804] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:00.119341 15844 raft_consensus.cc:3060] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:00.119499 23084 raft_consensus.cc:697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:06:00.120410 23084 consensus_queue.cc:237] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:06:00.111408 15202 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.117370 15154 raft_consensus.cc:3060] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:06:00.106781 23131 raft_consensus.cc:515] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.128332 23131 leader_election.cc:290] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.126232 15237 raft_consensus.cc:2468] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 23.
I20260709 13:06:00.130436 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8fe0d92f6f47dfb304a24bf915e2db" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:06:00.131057 15160 raft_consensus.cc:3060] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:00.132899 15810 leader_election.cc:304] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:06:00.126515 23137 leader_election.cc:290] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.135453 23131 raft_consensus.cc:2804] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:00.136225 15844 raft_consensus.cc:2468] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 19.
I20260709 13:06:00.137557 23137 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.137635 15201 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.138965 22958 raft_consensus.cc:2804] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:00.139667 23137 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.139710 22958 raft_consensus.cc:697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:06:00.141291 22958 consensus_queue.cc:237] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.143126 15163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "4b633b84351f40de9437e3d532fe6fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:06:00.144362 15154 raft_consensus.cc:2468] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b633b84351f40de9437e3d532fe6fa7 in term 27.
I20260709 13:06:00.146298 15160 raft_consensus.cc:2468] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 23.
I20260709 13:06:00.147917 15814 leader_election.cc:304] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
W20260709 13:06:00.126294 15813 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.136891 23131 raft_consensus.cc:697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:06:00.151024 23140 raft_consensus.cc:2804] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:00.151551 23140 raft_consensus.cc:697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:06:00.152486 23140 consensus_queue.cc:237] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.163980 23131 consensus_queue.cc:237] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.161552 15203 leader_election.cc:304] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
W20260709 13:06:00.166332 15813 leader_election.cc:336] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:00.167373 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:00.169099 23139 raft_consensus.cc:2804] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:06:00.169731 23139 raft_consensus.cc:697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 27 LEADER]: Becoming Leader. State: Replica: 4b633b84351f40de9437e3d532fe6fa7, State: Running, Role: LEADER
I20260709 13:06:00.170651 23139 consensus_queue.cc:237] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:06:00.177521 15202 leader_election.cc:336] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.147214 23137 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:00.183703 15813 leader_election.cc:336] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:00.184829 15202 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.185338 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:00.186504 23137 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:00.197250 23136 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.198069 23136 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.202649 23136 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:00.206494 15813 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.213960 15154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:06:00.215963 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:00.217308 23136 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:06:00.228583 23130 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.229152 23130 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.234573 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
I20260709 13:06:00.235163 15844 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 24.
I20260709 13:06:00.236407 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:06:00.243548 23121 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:06:00.244117 23121 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.244588 23121 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:00.238337 23130 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
I20260709 13:06:00.293615 23121 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.300345 23121 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:06:00.305351 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:00.303391 14985 catalog_manager.cc:5697] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 22 to 23, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 23 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:00.319499 15127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:34641, user_credentials={real_user=slave}} blocked reactor thread for 55188.2us
W20260709 13:06:00.319528 15128 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.323379 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "661df855a98f493ca654519f6be2164f" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82"
I20260709 13:06:00.323997 15844 raft_consensus.cc:3060] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:06:00.327143 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.327661 15128 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.331204 15844 raft_consensus.cc:2468] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 25.
I20260709 13:06:00.336448 15127 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.338635 23130 raft_consensus.cc:2804] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:06:00.339177 23130 raft_consensus.cc:697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 25 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:06:00.340106 23130 consensus_queue.cc:237] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
W20260709 13:06:00.358407 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.360461 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:00.384166 14979 catalog_manager.cc:5697] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 24 to 25, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 25 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:00.408265 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.421990 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.430197 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:00.432391 15154 raft_consensus.cc:1275] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:00.441929 23139 consensus_queue.cc:1048] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:00.442106 23130 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.442896 23130 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.445794 23130 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196:34641)
W20260709 13:06:00.446940 13967 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 15109)
I20260709 13:06:00.448037 15844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" is_pre_election: true
W20260709 13:06:00.452008 15128 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.452531 15128 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115, 3c28be6f86ea4fb6b79a47d3999f5a82
I20260709 13:06:00.453588 23151 raft_consensus.cc:2749] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:00.462803 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.490566 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.493947 23112 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.494560 23112 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.497596 23151 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.498070 15160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d" is_pre_election: true
I20260709 13:06:00.498080 23151 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.498920 15160 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 23.
I20260709 13:06:00.502235 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 
I20260709 13:06:00.503614 15237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7" is_pre_election: true
I20260709 13:06:00.504199 15237 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 22.
I20260709 13:06:00.506299 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 
I20260709 13:06:00.511602 23152 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:00.512004 23152 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.512349 23152 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:00.517004 23136 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:06:00.517379 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.517553 23136 raft_consensus.cc:493] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12daa67457954a0cab27e32bf85fa115)
I20260709 13:06:00.518060 23136 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:00.526238 15128 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.526540 14985 catalog_manager.cc:5697] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 22 to 23, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 23 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.506428 23112 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.506572 23151 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:06:00.535581 14983 catalog_manager.cc:5697] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 16 to 17, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 17 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.537917 14983 catalog_manager.cc:5697] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 18 to 19, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 19 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.539438 14983 catalog_manager.cc:5697] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 reported cstate change: term changed from 26 to 27, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195). New cstate: current_term: 27 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.552956 15163 raft_consensus.cc:1275] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:00.553289 23152 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:00.558995 23139 consensus_queue.cc:1048] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:00.566071 23136 raft_consensus.cc:515] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.569950 23136 leader_election.cc:290] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:00.572345 15156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16dab822009a4f00a0a480a48136ae25" candidate_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3d45101e0851484690b4f03a4dcdb63d"
I20260709 13:06:00.573050 15156 raft_consensus.cc:3060] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:00.578078 15163 raft_consensus.cc:1275] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 27 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:06:00.581059 23139 consensus_queue.cc:1048] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:06:00.581801 15813 leader_election.cc:336] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.584367 15156 raft_consensus.cc:2468] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c28be6f86ea4fb6b79a47d3999f5a82 in term 24.
W20260709 13:06:00.584668 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.587982 15814 leader_election.cc:304] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82, 3d45101e0851484690b4f03a4dcdb63d; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.595963 15228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740a0a73cde54c9abf2460f0bbdcf02d" candidate_uuid: "3d45101e0851484690b4f03a4dcdb63d" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4b633b84351f40de9437e3d532fe6fa7"
I20260709 13:06:00.596624 15228 raft_consensus.cc:3060] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:06:00.605811 15128 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:00.602706 23131 raft_consensus.cc:2804] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:00.601418 23152 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Requested vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:06:00.617419 23131 raft_consensus.cc:697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 LEADER]: Becoming Leader. State: Replica: 3c28be6f86ea4fb6b79a47d3999f5a82, State: Running, Role: LEADER
I20260709 13:06:00.618206 15228 raft_consensus.cc:2468] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d45101e0851484690b4f03a4dcdb63d in term 23.
I20260709 13:06:00.618917 23131 consensus_queue.cc:237] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:00.619518 15126 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115
I20260709 13:06:00.620424 23151 raft_consensus.cc:2804] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:00.621203 23151 raft_consensus.cc:697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Becoming Leader. State: Replica: 3d45101e0851484690b4f03a4dcdb63d, State: Running, Role: LEADER
I20260709 13:06:00.622092 23151 consensus_queue.cc:237] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
W20260709 13:06:00.623180 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.635809 15228 raft_consensus.cc:1275] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:00.641633 23107 consensus_queue.cc:1048] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:06:00.682483 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.689986 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.713503 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:00.714568 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:00.714998 14983 catalog_manager.cc:5697] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 reported cstate change: term changed from 23 to 24, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3c28be6f86ea4fb6b79a47d3999f5a82 (127.13.163.196). New cstate: current_term: 24 leader_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.720409 15160 raft_consensus.cc:1275] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:00.720539 15845 raft_consensus.cc:1275] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Refusing update from remote peer 4b633b84351f40de9437e3d532fe6fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:00.722103 23112 consensus_queue.cc:1048] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:06:00.723748 23120 consensus_queue.cc:1048] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:00.736160 14979 catalog_manager.cc:5697] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d reported cstate change: term changed from 22 to 23, leader changed from 12daa67457954a0cab27e32bf85fa115 (127.13.163.193) to 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194). New cstate: current_term: 23 leader_uuid: "3d45101e0851484690b4f03a4dcdb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:00.756883 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.924994 15845 raft_consensus.cc:1275] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:06:00.925945 15128 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.926908 23151 consensus_queue.cc:1048] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:06:01.031149 15228 raft_consensus.cc:1275] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Refusing update from remote peer 3d45101e0851484690b4f03a4dcdb63d: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:06:01.032800 23152 consensus_queue.cc:1048] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:06:01.036180 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:01.072253 15159 raft_consensus.cc:1275] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Refusing update from remote peer 3c28be6f86ea4fb6b79a47d3999f5a82: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:06:01.097672 23136 consensus_queue.cc:1048] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:06:01.111003 15128 consensus_peers.cc:597] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.141572 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.174644 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.214217 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:01.399920 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.447409 13967 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:01.524727 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.546644 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.560441 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.613198 15813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111) [suppressed 403 similar messages]
W20260709 13:06:01.633181 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:01.726243 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.894711 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:01.911369 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:01.932524 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:01.972888 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:02.030603 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:02.042405 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:02.169390 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:02.175508 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.248641 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:02.269292 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.275121 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.363121 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.375653 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:02.396771 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:02.448145 13967 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:02.467235 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.476605 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.612035 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.644213 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.823755 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.832858 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.874197 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.942790 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:02.989679 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.001081 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.023703 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.034386 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.080667 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.093611 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.196094 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.210897 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.216203 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.237578 15128 consensus_peers.cc:597] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.326602 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.384119 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.448695 13967 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:03.474398 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.502687 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.506768 15128 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:03.696054 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.822645 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.853708 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:03.877139 15126 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 43600.4us
W20260709 13:06:03.984325 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.097656 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.176648 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.189666 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.220674 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.287400 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.373445 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.401969 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.449470 13967 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:04.488792 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:04.508610 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.544134 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.653105 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.664915 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.675845 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.704385 15810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:42229, user_credentials={real_user=slave}} blocked reactor thread for 58493.6us
W20260709 13:06:04.765044 15813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:38845, user_credentials={real_user=slave}} blocked reactor thread for 89219.9us
W20260709 13:06:04.827315 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:04.842130 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:04.858450 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.908784 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:04.966985 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.020517 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.101542 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.142590 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.207508 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:05.247574 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.285632 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.336359 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.338629 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.398777 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.430968 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.450099 13967 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:05.459354 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.618396 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.660056 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.681392 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.737460 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:05.757609 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.769820 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.826819 15128 consensus_peers.cc:597] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.840652 15128 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:05.859007 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:06.015282 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:06.054975 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.155992 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.250190 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:06.400698 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.451303 13967 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:06.523113 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.584072 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.595324 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:06.618060 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.636720 15128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111) [suppressed 451 similar messages]
W20260709 13:06:06.649765 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.664953 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.746335 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:06.797816 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.898461 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:06.974583 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:07.017504 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.046697 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.068406 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.132858 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.139115 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:07.163420 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:07.186847 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:07.289841 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.374724 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:07.452572 13967 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:07.481994 14970 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 301fa97ba426402092eaf85685303b33 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 4b633b84351f40de9437e3d532fe6fa7 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 20 leader_uuid: "4b633b84351f40de9437e3d532fe6fa7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } } }
W20260709 13:06:07.497040 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.579883 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.832070 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.854425 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:07.879765 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.883152 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.885823 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.886890 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.893121 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.905867 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:07.994776 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.006922 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.027657 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.043938 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.227367 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.264951 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.265348 15128 consensus_peers.cc:597] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.339504 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.453130 13967 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 15109)
W20260709 13:06:08.482892 15128 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.485740 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.529840 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.752178 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800367d60 after lost signal to thread 13970
W20260709 13:06:08.755895 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800373ea0 after lost signal to thread 15020
W20260709 13:06:08.757642 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800091520 after lost signal to thread 15187
W20260709 13:06:08.758855 15017 debug-util.cc:398] Leaking SignalData structure 0x7b0800067b60 after lost signal to thread 15261
W20260709 13:06:08.760121 15017 debug-util.cc:398] Leaking SignalData structure 0x7b08002a94e0 after lost signal to thread 15872
W20260709 13:06:08.942946 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.945787 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.947973 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:08.948460 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:08.987273 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:09.099718 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.127601 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.174085 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.244665 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.300523 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.315368 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.335367 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.396452 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.452828 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.491371 15128 consensus_peers.cc:597] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.494743 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.519295 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.542285 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.592885 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:09.779639 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08001b2760 after lost signal to thread 13970
W20260709 13:06:09.780864 15184 debug-util.cc:398] Leaking SignalData structure 0x7b080017e200 after lost signal to thread 15020
W20260709 13:06:09.782312 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800277ac0 after lost signal to thread 15187
W20260709 13:06:09.783531 15184 debug-util.cc:398] Leaking SignalData structure 0x7b08000912a0 after lost signal to thread 15261
W20260709 13:06:09.784751 15184 debug-util.cc:398] Leaking SignalData structure 0x7b0800373c80 after lost signal to thread 15872
W20260709 13:06:09.885543 15128 consensus_peers.cc:597] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:09.915179 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:09.965312 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:10.142678 15128 consensus_peers.cc:597] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.146123 15128 consensus_peers.cc:597] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.286156 15128 consensus_peers.cc:597] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.292169 15128 consensus_peers.cc:597] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.427461 15128 consensus_peers.cc:597] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.531159 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:10.531795 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.543011 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.586355 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.609122 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.615464 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:10.829650 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.830295 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:10.889671 15128 consensus_peers.cc:597] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:10.924422 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:10.931875 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:10.951021 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:10.993016 15128 consensus_peers.cc:597] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:11.079217 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
W20260709 13:06:11.153112 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.155242 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.225178 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:11.365880 15128 consensus_peers.cc:597] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.406608 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:11.413909 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.448935 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.564867 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:11.571019 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
W20260709 13:06:11.583974 15203 consensus_peers.cc:597] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.622141 15814 consensus_peers.cc:597] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.660152 15203 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 423 similar messages]
W20260709 13:06:11.663970 15203 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.664711 15203 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.668200 15203 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.684093 15814 consensus_peers.cc:597] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.684970 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:11.690527 15203 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.694660 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.698962 15203 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.709726 15203 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.718276 15814 consensus_peers.cc:597] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.741331 15814 consensus_peers.cc:597] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.747701 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:11.748898 15203 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.796777 15814 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.799255 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.800391 13967 tablet_replica.cc:333] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.801506 13967 raft_consensus.cc:2243] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.803048 13967 raft_consensus.cc:2272] T d386fde0d10e4abf8b63290770e3ede0 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.805747 13967 tablet_replica.cc:333] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.806380 13967 raft_consensus.cc:2243] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.806905 13967 raft_consensus.cc:2272] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.809435 13967 tablet_replica.cc:333] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
W20260709 13:06:11.813020 15814 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.822661 13967 raft_consensus.cc:2243] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.823606 13967 raft_consensus.cc:2272] T 639c51e1c4da436b99c010fb41cebc63 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.824813 15203 consensus_peers.cc:597] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.826628 13967 tablet_replica.cc:333] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.827373 13967 raft_consensus.cc:2243] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.828025 13967 raft_consensus.cc:2272] T 7f14609939d04f1e90837cb17fe19457 P 3d45101e0851484690b4f03a4dcdb63d [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.830227 13967 tablet_replica.cc:333] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
W20260709 13:06:11.835945 15814 consensus_peers.cc:597] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.838039 13967 raft_consensus.cc:2243] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.839239 15203 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.848567 15814 consensus_peers.cc:597] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.850054 15814 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.839819 13967 raft_consensus.cc:2272] T 07a1de4f2ab24624aa87e5619961710d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.853787 15203 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.857604 13967 tablet_replica.cc:333] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.869583 13967 raft_consensus.cc:2243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.871035 13967 raft_consensus.cc:2272] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.874122 15203 consensus_peers.cc:597] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.874783 13967 tablet_replica.cc:333] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.875820 13967 raft_consensus.cc:2243] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.876631 13967 raft_consensus.cc:2272] T 42a84a565612402dbb559ec973692018 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.879891 13967 tablet_replica.cc:333] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
W20260709 13:06:11.884388 15814 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.896927 13967 raft_consensus.cc:2243] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.902308 15814 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.905632 13967 raft_consensus.cc:2272] T af13e9e38f6a4785bef931855aafd50c P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.911536 13967 tablet_replica.cc:333] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.912498 13967 raft_consensus.cc:2243] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.919546 15814 consensus_peers.cc:597] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.921878 15814 consensus_peers.cc:597] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.925292 15814 consensus_peers.cc:597] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.913755 13967 raft_consensus.cc:2272] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.929457 13967 tablet_replica.cc:333] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.931025 13967 raft_consensus.cc:2243] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.934768 15814 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.935608 13967 raft_consensus.cc:2272] T d4010b839395412ca61c77e701fcb965 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.935911 15203 consensus_peers.cc:597] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.939675 15203 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.946452 15128 consensus_peers.cc:597] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:11.946709 13967 tablet_replica.cc:333] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.947623 13967 raft_consensus.cc:2243] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.948465 13967 raft_consensus.cc:2272] T 4150ed0096c74899b55a5128304e91f6 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.951679 13967 tablet_replica.cc:333] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.955518 13967 raft_consensus.cc:2243] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.956354 13967 raft_consensus.cc:2272] T ec09a7a4d59c4296aa02dde6bd37f81c P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.959852 13967 tablet_replica.cc:333] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.960915 13967 raft_consensus.cc:2243] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.961591 13967 raft_consensus.cc:2272] T 9a20274fd45144db87032cdf948f324d P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.964771 13967 tablet_replica.cc:333] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.965734 13967 raft_consensus.cc:2243] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.970104 15814 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.970877 13967 raft_consensus.cc:2272] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.975292 15128 consensus_peers.cc:597] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:11.975553 13967 tablet_replica.cc:333] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
W20260709 13:06:11.989677 15814 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.978156 13967 raft_consensus.cc:2243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.991948 13967 raft_consensus.cc:2272] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.995146 13967 tablet_replica.cc:333] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:11.996008 13967 raft_consensus.cc:2243] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.997229 13967 raft_consensus.cc:2272] T 0afe5ac407a04216b625837a5bca525f P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.001150 13967 tablet_replica.cc:333] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.002022 13967 raft_consensus.cc:2243] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.002810 13967 raft_consensus.cc:2272] T 301fa97ba426402092eaf85685303b33 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.005563 13967 tablet_replica.cc:333] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.006460 13967 raft_consensus.cc:2243] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.007174 13967 raft_consensus.cc:2272] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.013267 15814 consensus_peers.cc:597] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:12.016100 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:12.019055 13967 tablet_replica.cc:333] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.019918 13967 raft_consensus.cc:2243] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.021183 13967 raft_consensus.cc:2272] T 447fc877e0f944fbb59ddd85ad25d8a9 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.023898 13967 tablet_replica.cc:333] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.024701 13967 raft_consensus.cc:2243] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.025936 13967 raft_consensus.cc:2272] T cf9063404f3d4db9b81ce66060eee5e7 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.028596 13967 tablet_replica.cc:333] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.029582 13967 raft_consensus.cc:2243] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.035109 13967 raft_consensus.cc:2272] T c84610e2d65945cab0886909d43d5e9e P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.038570 13967 tablet_replica.cc:333] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.039613 13967 raft_consensus.cc:2243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.041368 13967 raft_consensus.cc:2272] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.044652 13967 tablet_replica.cc:333] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.045589 13967 raft_consensus.cc:2243] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.046234 13967 raft_consensus.cc:2272] T 41c6ed4c81c041d18f143b6cd51bea35 P 3d45101e0851484690b4f03a4dcdb63d [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.054688 13967 tablet_replica.cc:333] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.055583 13967 raft_consensus.cc:2243] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.058730 13967 raft_consensus.cc:2272] T 365e6a51b3034d0483fdd8fc4cd362ea P 3d45101e0851484690b4f03a4dcdb63d [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.062165 13967 tablet_replica.cc:333] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.067595 13967 raft_consensus.cc:2243] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.068397 13967 raft_consensus.cc:2272] T bf60e792da464094a6aa2bfbcc5d71f2 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.098543 13967 tablet_replica.cc:333] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.100821 13967 raft_consensus.cc:2243] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.101963 13967 raft_consensus.cc:2272] T fe0a67b27bdc4c72abddb618266992a8 P 3d45101e0851484690b4f03a4dcdb63d [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.105777 13967 tablet_replica.cc:333] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.106632 13967 raft_consensus.cc:2243] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.119302 13967 raft_consensus.cc:2272] T ee2d00b915f04988a8794ab25f22cbd5 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.122503 13967 tablet_replica.cc:333] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.147086 13967 raft_consensus.cc:2243] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.147888 13967 raft_consensus.cc:2272] T 38a7fd74046f492f8ef8366fbc675ee0 P 3d45101e0851484690b4f03a4dcdb63d [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.166878 13967 tablet_replica.cc:333] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.167953 13967 raft_consensus.cc:2243] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.168709 13967 raft_consensus.cc:2272] T 4d62d65616394d059c479b07e82eccb9 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.171339 13967 tablet_replica.cc:333] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
W20260709 13:06:12.184916 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:12.172408 13967 raft_consensus.cc:2243] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.189622 13967 raft_consensus.cc:2272] T 1868a2755dea467c9c719191d9927234 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.194249 13967 tablet_replica.cc:333] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.195055 13967 raft_consensus.cc:2243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.207263 13967 raft_consensus.cc:2272] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.210515 13967 tablet_replica.cc:333] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.212101 13967 raft_consensus.cc:2243] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.213606 13967 raft_consensus.cc:2272] T 661df855a98f493ca654519f6be2164f P 3d45101e0851484690b4f03a4dcdb63d [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.214262 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:12.222828 13967 tablet_replica.cc:333] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.223637 13967 raft_consensus.cc:2243] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.224308 13967 raft_consensus.cc:2272] T 15cd34bca3be499d87220d6e1b2a763f P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.227169 13967 tablet_replica.cc:333] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.227909 13967 raft_consensus.cc:2243] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.229084 13967 raft_consensus.cc:2272] T 211046e319804173a9b8954112b8d044 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.232062 13967 tablet_replica.cc:333] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.232832 13967 raft_consensus.cc:2243] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.242038 13967 raft_consensus.cc:2272] T b358d067cea0428190a484601dea7c69 P 3d45101e0851484690b4f03a4dcdb63d [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.245760 13967 tablet_replica.cc:333] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.246553 13967 raft_consensus.cc:2243] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.247202 13967 raft_consensus.cc:2272] T 2b4be666e4a740bd853d8f2748ad6191 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.265595 13967 tablet_replica.cc:333] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.266472 13967 raft_consensus.cc:2243] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.267231 13967 raft_consensus.cc:2272] T 16dab822009a4f00a0a480a48136ae25 P 3d45101e0851484690b4f03a4dcdb63d [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.269913 13967 tablet_replica.cc:333] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.270730 13967 raft_consensus.cc:2243] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.271422 13967 raft_consensus.cc:2272] T f11eb27eddd0475ca037593dcdb66376 P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.274152 13967 tablet_replica.cc:333] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.274910 13967 raft_consensus.cc:2243] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.276314 13967 raft_consensus.cc:2272] T 740a0a73cde54c9abf2460f0bbdcf02d P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.291882 13967 tablet_replica.cc:333] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.292717 13967 raft_consensus.cc:2243] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.293910 13967 raft_consensus.cc:2272] T f52fa48697b145c98311ef7704e35002 P 3d45101e0851484690b4f03a4dcdb63d [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.300897 13967 tablet_replica.cc:333] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.301693 13967 raft_consensus.cc:2243] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.302356 13967 raft_consensus.cc:2272] T eb265c17a8bf4acfb7e480e4cf91d39b P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.305091 13967 tablet_replica.cc:333] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.305747 13967 raft_consensus.cc:2243] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.306486 13967 raft_consensus.cc:2272] T ff3342a5b2ad49e2b252d7cd47b9851d P 3d45101e0851484690b4f03a4dcdb63d [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.309095 13967 tablet_replica.cc:333] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.309749 13967 raft_consensus.cc:2243] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.310482 13967 raft_consensus.cc:2272] T efef46e0b3c14af2a7179beba5c160b4 P 3d45101e0851484690b4f03a4dcdb63d [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.313482 13967 tablet_replica.cc:333] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d: stopping tablet replica
I20260709 13:06:12.314379 13967 raft_consensus.cc:2243] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.315584 13967 raft_consensus.cc:2272] T 5e777c555e1b4153913b5e6459164454 P 3d45101e0851484690b4f03a4dcdb63d [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.378757 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.554620 15202 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.555544 15202 consensus_peers.cc:597] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.556049 15202 consensus_peers.cc:597] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.566358 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:12.651809 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.678040 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:12.931072 15202 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:12.979229 23206 raft_consensus.cc:493] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:12.979842 23206 raft_consensus.cc:515] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:12.982735 23206 leader_election.cc:290] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.111539 15202 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.120889 23218 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.121688 23218 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.125056 23218 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.143800 15814 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.162253 15203 leader_election.cc:336] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.162868 15203 leader_election.cc:304] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.164608 23215 raft_consensus.cc:2749] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:13.168970 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:13.171310 15813 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.172652 15813 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.174494 23220 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:13.242247 23219 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.243224 23219 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.250605 23219 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.251075 15814 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.255965 15813 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:13.258215 15202 consensus_peers.cc:597] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:13.258378 15813 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.259428 23219 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:13.264791 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:13.268294 15202 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:06:13.279606 23216 raft_consensus.cc:493] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.280256 23216 raft_consensus.cc:515] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:13.285472 23219 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.286084 23219 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.289914 23219 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
W20260709 13:06:13.294858 15814 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.296892 15203 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.297946 23215 raft_consensus.cc:493] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
W20260709 13:06:13.298740 15813 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.298463 23215 raft_consensus.cc:515] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:13.299178 15813 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.300422 23220 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:13.299721 23219 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.300985 23219 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.302037 23215 leader_election.cc:290] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.308645 15814 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.309669 15203 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.310160 23216 leader_election.cc:290] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.319545 15202 leader_election.cc:336] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.320091 15202 leader_election.cc:304] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.321753 23216 raft_consensus.cc:2749] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:13.322571 15202 leader_election.cc:336] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.323083 15202 leader_election.cc:304] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.326335 23216 raft_consensus.cc:2749] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:13.330090 23219 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.330461 15813 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.331110 15813 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.331705 23219 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:06:13.361822 15202 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:13.378398 23208 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.379002 23208 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.382714 23208 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.387166 15813 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:13.390512 15814 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.392191 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.393090 23208 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:13.394436 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:13.395318 23208 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.395833 23208 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.397581 23208 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.403852 15814 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.406245 15813 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.406617 15813 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.407186 23208 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:13.431098 23215 raft_consensus.cc:493] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.431641 23215 raft_consensus.cc:515] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:13.433703 23215 leader_election.cc:290] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.439177 15202 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:13.441383 15203 leader_election.cc:336] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.441778 15203 leader_election.cc:304] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.442529 23215 raft_consensus.cc:2749] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:13.488404 23215 raft_consensus.cc:493] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.488969 23215 raft_consensus.cc:515] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } }
I20260709 13:06:13.491003 23215 leader_election.cc:290] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.497145 15203 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:13.499277 15202 leader_election.cc:336] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:13.500284 15202 leader_election.cc:304] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b633b84351f40de9437e3d532fe6fa7; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.501099 23215 raft_consensus.cc:2749] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:13.506889 15202 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:13.555928 23219 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:13.556485 23219 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:13.558815 23219 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:13.563180 15813 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:13.565253 15814 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:13.565778 15814 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:13.567648 23219 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:13.659957 15202 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:13.669809 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:13.771564 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:13.789631 15202 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:13.781402 15203 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:13.801049 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.039263 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.051362 15814 consensus_peers.cc:597] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.055493 15814 consensus_peers.cc:597] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.080823 15203 consensus_peers.cc:597] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.089087 15203 consensus_peers.cc:597] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.103281 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08001de020 after lost signal to thread 13970
W20260709 13:06:14.105365 15869 debug-util.cc:398] Leaking SignalData structure 0x7b08003aa9a0 after lost signal to thread 15020
W20260709 13:06:14.107631 15869 debug-util.cc:398] Leaking SignalData structure 0x7b0800318f80 after lost signal to thread 15261
W20260709 13:06:14.109505 15869 debug-util.cc:398] Leaking SignalData structure 0x7b0800319120 after lost signal to thread 15872
W20260709 13:06:14.135917 15814 consensus_peers.cc:597] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:14.158278 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
W20260709 13:06:14.166749 15203 consensus_peers.cc:597] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.188796 15203 consensus_peers.cc:597] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.198297 15814 consensus_peers.cc:597] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.217257 15203 consensus_peers.cc:597] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.222793 15203 consensus_peers.cc:597] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.236093 15814 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.244151 15814 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.265991 15202 consensus_peers.cc:597] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:14.266732 15202 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:14.295758 15203 consensus_peers.cc:597] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.296085 15814 consensus_peers.cc:597] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.303687 15814 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.304266 15813 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.304330 15814 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.323889 15203 consensus_peers.cc:597] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.337464 15813 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.331300 15814 consensus_peers.cc:597] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.339841 15203 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.344159 15202 consensus_peers.cc:597] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:14.352614 15202 consensus_peers.cc:597] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.402637 15814 consensus_peers.cc:597] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.403494 15814 consensus_peers.cc:597] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.396808 15202 consensus_peers.cc:597] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:14.433167 15203 consensus_peers.cc:597] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.456697 15203 consensus_peers.cc:597] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:14.459980 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
W20260709 13:06:14.466485 15814 consensus_peers.cc:597] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.471671 15202 consensus_peers.cc:597] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:14.483896 15814 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.471145 15203 consensus_peers.cc:597] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.486429 15814 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.489161 15814 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.494431 15203 consensus_peers.cc:597] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.503921 15810 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.505682 15810 consensus_peers.cc:597] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.506500 15810 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.507022 15810 consensus_peers.cc:597] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.507678 15810 consensus_peers.cc:597] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.524605 15814 consensus_peers.cc:597] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:14.531962 15813 consensus_peers.cc:597] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:14.583143 15814 consensus_peers.cc:597] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:14.601414 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:14.604418 13967 tablet_replica.cc:333] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.605643 13967 raft_consensus.cc:2243] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:14.606362 15810 consensus_peers.cc:597] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.607118 13967 raft_consensus.cc:2272] T d386fde0d10e4abf8b63290770e3ede0 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.609654 15810 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.610378 15810 consensus_peers.cc:597] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.610800 13967 tablet_replica.cc:333] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
W20260709 13:06:14.611130 15810 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.611924 13967 raft_consensus.cc:2243] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.613037 13967 raft_consensus.cc:2272] T 7f14609939d04f1e90837cb17fe19457 P 4b633b84351f40de9437e3d532fe6fa7 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.614429 15810 consensus_peers.cc:597] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.615222 15810 consensus_peers.cc:597] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.616176 13967 tablet_replica.cc:333] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.616992 13967 raft_consensus.cc:2243] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.617586 13967 raft_consensus.cc:2272] T 639c51e1c4da436b99c010fb41cebc63 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.620227 13967 tablet_replica.cc:333] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.621007 13967 raft_consensus.cc:2243] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.622208 13967 raft_consensus.cc:2272] T 42a84a565612402dbb559ec973692018 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.624818 13967 tablet_replica.cc:333] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.625489 13967 raft_consensus.cc:2243] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.626318 13967 raft_consensus.cc:2272] T 30853833ff254a12bcc77a177a964bb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.628808 13967 tablet_replica.cc:333] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.629719 13967 raft_consensus.cc:2243] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.631224 13967 raft_consensus.cc:2272] T 0a5f358b91ee427880ebecaca8118e4f P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.634145 13967 tablet_replica.cc:333] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.634915 13967 raft_consensus.cc:2243] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.636241 13967 raft_consensus.cc:2272] T d4010b839395412ca61c77e701fcb965 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.639142 13967 tablet_replica.cc:333] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.639962 13967 raft_consensus.cc:2243] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.640767 13967 raft_consensus.cc:2272] T de505aa1e895447486f53629c5269998 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.643141 13967 tablet_replica.cc:333] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.643908 13967 raft_consensus.cc:2243] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.644747 13967 raft_consensus.cc:2272] T af12fc69de0b47b9adaf0267fb143776 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.647675 13967 tablet_replica.cc:333] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.648474 13967 raft_consensus.cc:2243] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.649248 13967 raft_consensus.cc:2272] T 4150ed0096c74899b55a5128304e91f6 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.652169 13967 tablet_replica.cc:333] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.652936 13967 raft_consensus.cc:2243] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.654093 13967 raft_consensus.cc:2272] T 08bf5d5f44ec498499e1cb1be55c2283 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.656776 13967 tablet_replica.cc:333] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.657670 13967 raft_consensus.cc:2243] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.658478 13967 raft_consensus.cc:2272] T ec09a7a4d59c4296aa02dde6bd37f81c P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.661497 13967 tablet_replica.cc:333] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.662240 13967 raft_consensus.cc:2243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.663201 13967 raft_consensus.cc:2272] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.666323 13967 tablet_replica.cc:333] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.667213 13967 raft_consensus.cc:2243] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.668006 13967 raft_consensus.cc:2272] T f52fa48697b145c98311ef7704e35002 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.670835 13967 tablet_replica.cc:333] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.671657 13967 raft_consensus.cc:2243] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.673135 13967 raft_consensus.cc:2272] T 6b6a82b88a6f4f6093256a0d85708d37 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.675796 13967 tablet_replica.cc:333] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.676563 13967 raft_consensus.cc:2243] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.677476 13967 raft_consensus.cc:2272] T 73ae83bb85154bb08a2889f0f9f99cd8 P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.680402 13967 tablet_replica.cc:333] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.681193 13967 raft_consensus.cc:2243] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.682301 13967 raft_consensus.cc:2272] T 41c6ed4c81c041d18f143b6cd51bea35 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.685067 13967 tablet_replica.cc:333] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.685782 13967 raft_consensus.cc:2243] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.687412 13967 raft_consensus.cc:2272] T 9a20274fd45144db87032cdf948f324d P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.690199 13967 tablet_replica.cc:333] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
W20260709 13:06:14.690771 15810 consensus_peers.cc:597] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.690997 13967 raft_consensus.cc:2243] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.692261 13967 raft_consensus.cc:2272] T 808dd81f1b604329848aa94e3ecedc36 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.695067 13967 tablet_replica.cc:333] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.695812 13967 raft_consensus.cc:2243] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.698323 13967 raft_consensus.cc:2272] T 301fa97ba426402092eaf85685303b33 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.700925 13967 tablet_replica.cc:333] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.701732 13967 raft_consensus.cc:2243] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.702921 13967 raft_consensus.cc:2272] T 3e0c065a7fec4f5c88b4bc85b3508231 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.705986 13967 tablet_replica.cc:333] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.706877 13967 raft_consensus.cc:2243] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.709918 13967 raft_consensus.cc:2272] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.712826 13967 tablet_replica.cc:333] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.713667 13967 raft_consensus.cc:2243] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:06:14.714079 15810 consensus_peers.cc:597] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.714726 13967 raft_consensus.cc:2272] T af13e9e38f6a4785bef931855aafd50c P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.719133 13967 tablet_replica.cc:333] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.719928 13967 raft_consensus.cc:2243] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.721161 13967 raft_consensus.cc:2272] T c84610e2d65945cab0886909d43d5e9e P 4b633b84351f40de9437e3d532fe6fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.724020 13967 tablet_replica.cc:333] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.724789 13967 raft_consensus.cc:2243] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.725468 13967 raft_consensus.cc:2272] T 447fc877e0f944fbb59ddd85ad25d8a9 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.728364 13967 tablet_replica.cc:333] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.729163 13967 raft_consensus.cc:2243] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.730109 13967 raft_consensus.cc:2272] T eaa49b1ae77944d7b6866e819d63d466 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.733069 13967 tablet_replica.cc:333] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.734036 13967 raft_consensus.cc:2243] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.735163 13967 raft_consensus.cc:2272] T ee2d00b915f04988a8794ab25f22cbd5 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.738260 13967 tablet_replica.cc:333] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
W20260709 13:06:14.746141 15810 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.748600 15813 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:14.749303 15810 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.750784 13967 raft_consensus.cc:2243] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.752140 13967 raft_consensus.cc:2272] T 38a7fd74046f492f8ef8366fbc675ee0 P 4b633b84351f40de9437e3d532fe6fa7 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.754925 13967 tablet_replica.cc:333] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.755712 13967 raft_consensus.cc:2243] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.756359 13967 raft_consensus.cc:2272] T 4d62d65616394d059c479b07e82eccb9 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.758271 15810 consensus_peers.cc:597] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.767946 13967 tablet_replica.cc:333] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.769148 13967 raft_consensus.cc:2243] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.769236 23219 raft_consensus.cc:493] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:14.770615 13967 raft_consensus.cc:2272] T 1868a2755dea467c9c719191d9927234 P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.769896 23219 raft_consensus.cc:515] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:14.773669 13967 tablet_replica.cc:333] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.774505 13967 raft_consensus.cc:2243] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.775686 13967 raft_consensus.cc:2272] T 2b4be666e4a740bd853d8f2748ad6191 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.778867 13967 tablet_replica.cc:333] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.779635 13967 raft_consensus.cc:2243] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.780491 13967 raft_consensus.cc:2272] T 14bd17b479354abca0014ae059714e8f P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.785650 15813 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:14.786147 13967 tablet_replica.cc:333] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
W20260709 13:06:14.785699 15810 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.786963 13967 raft_consensus.cc:2243] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.788179 13967 raft_consensus.cc:2272] T f11eb27eddd0475ca037593dcdb66376 P 4b633b84351f40de9437e3d532fe6fa7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.790406 23219 leader_election.cc:290] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:14.790894 13967 tablet_replica.cc:333] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.795399 13967 raft_consensus.cc:2243] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:14.795943 15814 leader_election.cc:336] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:14.796491 15814 leader_election.cc:304] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:14.797681 23219 raft_consensus.cc:2749] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:14.796123 13967 raft_consensus.cc:2272] T 740a0a73cde54c9abf2460f0bbdcf02d P 4b633b84351f40de9437e3d532fe6fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.802029 13967 tablet_replica.cc:333] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.802919 13967 raft_consensus.cc:2243] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.803622 13967 raft_consensus.cc:2272] T 30c8fd833bca41f69ed253599a8889b9 P 4b633b84351f40de9437e3d532fe6fa7 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.811167 15813 consensus_peers.cc:597] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:14.813177 13967 tablet_replica.cc:333] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.814262 13967 raft_consensus.cc:2243] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.815340 13967 raft_consensus.cc:2272] T 365e6a51b3034d0483fdd8fc4cd362ea P 4b633b84351f40de9437e3d532fe6fa7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.818338 13967 tablet_replica.cc:333] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.819110 13967 raft_consensus.cc:2243] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:06:14.837137 15813 consensus_peers.cc:597] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:14.842460 13967 raft_consensus.cc:2272] T ca6bbc5df3084af4a938113f16eeedb5 P 4b633b84351f40de9437e3d532fe6fa7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.845489 13967 tablet_replica.cc:333] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.846349 13967 raft_consensus.cc:2243] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.847793 13967 raft_consensus.cc:2272] T eb265c17a8bf4acfb7e480e4cf91d39b P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.850693 13967 tablet_replica.cc:333] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.851533 13967 raft_consensus.cc:2243] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.852800 13967 raft_consensus.cc:2272] T efef46e0b3c14af2a7179beba5c160b4 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.855656 13967 tablet_replica.cc:333] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.856428 13967 raft_consensus.cc:2243] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.857146 13967 raft_consensus.cc:2272] T 211046e319804173a9b8954112b8d044 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.858836 15810 consensus_peers.cc:597] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:14.860144 13967 tablet_replica.cc:333] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.861012 13967 raft_consensus.cc:2243] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.861853 13967 raft_consensus.cc:2272] T bf60e792da464094a6aa2bfbcc5d71f2 P 4b633b84351f40de9437e3d532fe6fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.864908 13967 tablet_replica.cc:333] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.865634 13967 raft_consensus.cc:2243] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.867012 13967 raft_consensus.cc:2272] T a12ada8057b141518113efeb519c6514 P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.874009 13967 tablet_replica.cc:333] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.874717 13967 raft_consensus.cc:2243] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.875315 13967 raft_consensus.cc:2272] T d50a21b24380415781c0e63c5d613aff P 4b633b84351f40de9437e3d532fe6fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.878026 13967 tablet_replica.cc:333] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7: stopping tablet replica
I20260709 13:06:14.878688 13967 raft_consensus.cc:2243] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.879691 13967 raft_consensus.cc:2272] T 06f7ae7c1d614611bb074a65fcc9a959 P 4b633b84351f40de9437e3d532fe6fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.902730 15810 consensus_peers.cc:597] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Couldn't send request to peer 4b633b84351f40de9437e3d532fe6fa7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:14.949003 15813 consensus_peers.cc:597] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:15.010278 23230 raft_consensus.cc:493] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.010572 23220 raft_consensus.cc:493] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.011104 23230 raft_consensus.cc:515] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.011332 23220 raft_consensus.cc:515] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.013684 23230 leader_election.cc:290] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
I20260709 13:06:15.013825 23220 leader_election.cc:290] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:15.020387 15813 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.021018 15813 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.026458 15814 leader_election.cc:336] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.026942 15814 leader_election.cc:304] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.027760 23220 raft_consensus.cc:2749] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:15.028750 15814 leader_election.cc:336] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.029206 15814 leader_election.cc:304] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.029919 23220 raft_consensus.cc:2749] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:15.061091 15813 consensus_peers.cc:597] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:15.216961 15813 consensus_peers.cc:597] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:15.230860 23219 raft_consensus.cc:493] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.231590 23219 raft_consensus.cc:515] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.234082 23219 leader_election.cc:290] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:15.239182 15813 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.245914 15814 leader_election.cc:336] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.246446 15814 leader_election.cc:304] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.247648 23220 raft_consensus.cc:2749] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:06:15.277099 23233 raft_consensus.cc:493] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.277849 23233 raft_consensus.cc:515] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.283056 23233 leader_election.cc:290] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845)
W20260709 13:06:15.288210 15813 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.292647 15814 leader_election.cc:336] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.298892 15814 leader_election.cc:304] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.300575 23233 raft_consensus.cc:2749] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:15.398370 23233 raft_consensus.cc:493] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.399066 23233 raft_consensus.cc:515] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.401618 23233 leader_election.cc:290] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:15.406688 15813 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.407501 15814 leader_election.cc:336] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.407976 15814 leader_election.cc:304] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.408758 23233 raft_consensus.cc:2749] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:15.437409 15813 consensus_peers.cc:597] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:15.629099 23233 raft_consensus.cc:493] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d45101e0851484690b4f03a4dcdb63d)
I20260709 13:06:15.629684 23233 raft_consensus.cc:515] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.631767 23233 leader_election.cc:290] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049)
W20260709 13:06:15.636500 15814 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:15.637094 15813 leader_election.cc:336] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:15.637552 15813 leader_election.cc:304] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 3d45101e0851484690b4f03a4dcdb63d
I20260709 13:06:15.638378 23233 raft_consensus.cc:2749] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:15.694025 15813 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:15.720314 23230 raft_consensus.cc:493] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.721004 23230 raft_consensus.cc:515] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.722792 23230 leader_election.cc:290] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:15.726472 15813 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.726976 15810 leader_election.cc:336] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:15.727428 15810 leader_election.cc:304] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:15.728181 23230 raft_consensus.cc:2749] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:15.728806 23230 raft_consensus.cc:493] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.729249 23230 raft_consensus.cc:515] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.730942 23230 leader_election.cc:290] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:06:15.732059 23230 raft_consensus.cc:493] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.732666 23230 raft_consensus.cc:515] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.735088 23230 leader_election.cc:290] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:15.736254 15814 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:15.739064 15810 leader_election.cc:336] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
W20260709 13:06:15.739217 15813 leader_election.cc:336] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:15.739588 15810 leader_election.cc:304] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:15.740473 23230 raft_consensus.cc:2749] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:15.743845 15810 leader_election.cc:336] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:15.744333 15810 leader_election.cc:304] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:06:15.744635 15813 consensus_peers.cc:597] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:15.745190 23233 raft_consensus.cc:2749] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:15.760273 23230 raft_consensus.cc:493] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.761094 23230 raft_consensus.cc:515] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.763600 23230 leader_election.cc:290] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:15.767535 15813 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
W20260709 13:06:15.768189 15810 leader_election.cc:336] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:15.768668 15810 leader_election.cc:304] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:15.769378 23233 raft_consensus.cc:2749] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.802012 23230 raft_consensus.cc:493] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.802515 23230 raft_consensus.cc:515] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.804312 23230 leader_election.cc:290] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:15.808687 15814 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:15.809418 15810 leader_election.cc:336] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:15.809966 15810 leader_election.cc:304] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:15.811072 23230 raft_consensus.cc:2749] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.988927 23230 raft_consensus.cc:493] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.989086 23233 raft_consensus.cc:493] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:15.989534 23230 raft_consensus.cc:515] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.989724 23233 raft_consensus.cc:515] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } attrs { replace: true } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.991731 23230 leader_election.cc:290] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
I20260709 13:06:15.992031 23233 leader_election.cc:290] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:15.996809 15810 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
W20260709 13:06:15.997648 15814 leader_election.cc:336] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
I20260709 13:06:15.997846 23233 raft_consensus.cc:493] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
W20260709 13:06:15.997396 15810 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:15.998129 15814 leader_election.cc:304] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
W20260709 13:06:15.998708 15813 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:15.998461 23233 raft_consensus.cc:515] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:15.998941 23230 raft_consensus.cc:2749] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.999794 15813 leader_election.cc:304] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:16.000537 23230 raft_consensus.cc:2749] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:16.000665 23233 leader_election.cc:290] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:16.000349 15814 leader_election.cc:336] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:16.004801 15810 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
W20260709 13:06:16.005360 15813 leader_election.cc:336] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:16.005739 15813 leader_election.cc:304] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:16.006345 23230 raft_consensus.cc:2749] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:16.043208 15813 consensus_peers.cc:597] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:06:16.187098 23230 raft_consensus.cc:493] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:16.187610 23230 raft_consensus.cc:515] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:16.189893 23230 leader_election.cc:290] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:16.193830 15810 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
W20260709 13:06:16.194742 15813 leader_election.cc:336] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:16.195125 15813 leader_election.cc:304] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:16.195717 23230 raft_consensus.cc:2749] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:16.204968 15813 consensus_peers.cc:597] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:16.208987 15813 consensus_peers.cc:597] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:16.222311 23230 raft_consensus.cc:493] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:16.222958 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:06:16.222959 23230 raft_consensus.cc:515] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d45101e0851484690b4f03a4dcdb63d" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 38049 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:16.225788 23230 leader_election.cc:290] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:16.230031 15814 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111)
W20260709 13:06:16.230530 15810 leader_election.cc:336] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:16.231086 15810 leader_election.cc:304] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 3d45101e0851484690b4f03a4dcdb63d, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:16.231863 23230 raft_consensus.cc:2749] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:16.281030 15813 consensus_peers.cc:597] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:16.323922 13967 tablet_server.cc:179] TabletServer@127.13.163.196:0 shutting down...
W20260709 13:06:16.377535 15813 consensus_peers.cc:597] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Couldn't send request to peer 12daa67457954a0cab27e32bf85fa115. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:16.382051 23233 raft_consensus.cc:493] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4b633b84351f40de9437e3d532fe6fa7)
I20260709 13:06:16.382725 23233 raft_consensus.cc:515] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "12daa67457954a0cab27e32bf85fa115" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 38845 } } peers { permanent_uuid: "4b633b84351f40de9437e3d532fe6fa7" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 42229 } } peers { permanent_uuid: "3c28be6f86ea4fb6b79a47d3999f5a82" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 34641 } attrs { promote: false } }
I20260709 13:06:16.383924 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:16.384961 13967 tablet_replica.cc:333] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.386169 13967 raft_consensus.cc:2243] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.387254 13967 raft_consensus.cc:2272] T de505aa1e895447486f53629c5269998 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.387985 23233 leader_election.cc:290] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845), 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229)
W20260709 13:06:16.390275 15814 consensus_peers.cc:597] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:16.390601 13967 tablet_replica.cc:333] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
W20260709 13:06:16.390784 15810 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:16.391686 13967 raft_consensus.cc:2243] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.392518 13967 raft_consensus.cc:2272] T ca6bbc5df3084af4a938113f16eeedb5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:16.393316 15813 leader_election.cc:336] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 12daa67457954a0cab27e32bf85fa115 (127.13.163.193:38845): Network error: Client connection negotiation failed: client connection to 127.13.163.193:38845: connect: Connection refused (error 111)
I20260709 13:06:16.393751 15813 leader_election.cc:304] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c28be6f86ea4fb6b79a47d3999f5a82; no voters: 12daa67457954a0cab27e32bf85fa115, 4b633b84351f40de9437e3d532fe6fa7
I20260709 13:06:16.394431 23233 raft_consensus.cc:2749] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:16.395380 13967 tablet_replica.cc:333] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.396106 13967 raft_consensus.cc:2243] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.396932 13967 raft_consensus.cc:2272] T 9cb4f228ea0b4cc6831089c6ddf8ec96 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.398998 13967 tablet_replica.cc:333] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.399780 13967 raft_consensus.cc:2243] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.400764 13967 raft_consensus.cc:2272] T 30853833ff254a12bcc77a177a964bb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.403976 13967 tablet_replica.cc:333] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.404626 13967 raft_consensus.cc:2243] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.405581 13967 raft_consensus.cc:2272] T fe0a67b27bdc4c72abddb618266992a8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.407841 13967 tablet_replica.cc:333] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.408406 13967 raft_consensus.cc:2243] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.409245 13967 raft_consensus.cc:2272] T 639c51e1c4da436b99c010fb41cebc63 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.411372 13967 tablet_replica.cc:333] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.411957 13967 raft_consensus.cc:2243] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.413039 13967 raft_consensus.cc:2272] T 4d62d65616394d059c479b07e82eccb9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.415768 13967 tablet_replica.cc:333] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.416260 13967 raft_consensus.cc:2243] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.417047 13967 raft_consensus.cc:2272] T 73ae83bb85154bb08a2889f0f9f99cd8 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.418896 13967 tablet_replica.cc:333] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.419409 13967 raft_consensus.cc:2243] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.420239 13967 raft_consensus.cc:2272] T 30c8fd833bca41f69ed253599a8889b9 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.422132 13967 tablet_replica.cc:333] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.422598 13967 raft_consensus.cc:2243] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.423084 13967 raft_consensus.cc:2272] T 661df855a98f493ca654519f6be2164f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.424935 13967 tablet_replica.cc:333] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.425511 13967 raft_consensus.cc:2243] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.426285 13967 raft_consensus.cc:2272] T ec09a7a4d59c4296aa02dde6bd37f81c P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.428308 13967 tablet_replica.cc:333] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.428828 13967 raft_consensus.cc:2243] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.429374 13967 raft_consensus.cc:2272] T 301fa97ba426402092eaf85685303b33 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.431278 13967 tablet_replica.cc:333] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.431819 13967 raft_consensus.cc:2243] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.432294 13967 raft_consensus.cc:2272] T 3e0c065a7fec4f5c88b4bc85b3508231 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.434559 13967 tablet_replica.cc:333] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.435443 13967 raft_consensus.cc:2243] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.436050 13967 raft_consensus.cc:2272] T 0afe5ac407a04216b625837a5bca525f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.438470 13967 tablet_replica.cc:333] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.439206 13967 raft_consensus.cc:2243] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.439857 13967 raft_consensus.cc:2272] T 808dd81f1b604329848aa94e3ecedc36 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.442444 13967 tablet_replica.cc:333] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.443012 13967 raft_consensus.cc:2243] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:06:16.443181 15814 consensus_peers.cc:597] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 -> Peer 3d45101e0851484690b4f03a4dcdb63d (127.13.163.194:38049): Couldn't send request to peer 3d45101e0851484690b4f03a4dcdb63d. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:38049: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:16.443907 13967 raft_consensus.cc:2272] T d50a21b24380415781c0e63c5d613aff P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.446221 13967 tablet_replica.cc:333] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.446806 13967 raft_consensus.cc:2243] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.447343 13967 raft_consensus.cc:2272] T a12ada8057b141518113efeb519c6514 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.449355 13967 tablet_replica.cc:333] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.449958 13967 raft_consensus.cc:2243] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.450407 13967 raft_consensus.cc:2272] T 06f7ae7c1d614611bb074a65fcc9a959 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.452775 13967 tablet_replica.cc:333] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.453545 13967 raft_consensus.cc:2243] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.454615 13967 raft_consensus.cc:2272] T 4150ed0096c74899b55a5128304e91f6 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.457232 13967 tablet_replica.cc:333] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.457945 13967 raft_consensus.cc:2243] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.458792 13967 raft_consensus.cc:2272] T 08bf5d5f44ec498499e1cb1be55c2283 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.460616 13967 tablet_replica.cc:333] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.461187 13967 raft_consensus.cc:2243] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.462177 13967 raft_consensus.cc:2272] T eaa49b1ae77944d7b6866e819d63d466 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.464138 13967 tablet_replica.cc:333] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.464689 13967 raft_consensus.cc:2243] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.465186 13967 raft_consensus.cc:2272] T 2b4be666e4a740bd853d8f2748ad6191 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.467152 13967 tablet_replica.cc:333] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.467753 13967 raft_consensus.cc:2243] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.468508 13967 raft_consensus.cc:2272] T 13e4a6d3ac0a4d6baf95fb7d5c21b5f7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.470319 13967 tablet_replica.cc:333] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.470911 13967 raft_consensus.cc:2243] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.471472 13967 raft_consensus.cc:2272] T c84610e2d65945cab0886909d43d5e9e P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.473985 13967 tablet_replica.cc:333] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.474551 13967 raft_consensus.cc:2243] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.475127 13967 raft_consensus.cc:2272] T cf9063404f3d4db9b81ce66060eee5e7 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.477677 13967 tablet_replica.cc:333] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.478349 13967 raft_consensus.cc:2243] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.478768 13967 raft_consensus.cc:2272] T 60df009dd21a4cb3867d92cb3a69c1e4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.480464 13967 tablet_replica.cc:333] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.481040 13967 raft_consensus.cc:2243] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.481848 13967 raft_consensus.cc:2272] T af12fc69de0b47b9adaf0267fb143776 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.484103 13967 tablet_replica.cc:333] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.484855 13967 raft_consensus.cc:2243] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.485875 13967 raft_consensus.cc:2272] T bf60e792da464094a6aa2bfbcc5d71f2 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.488257 13967 tablet_replica.cc:333] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.489053 13967 raft_consensus.cc:2243] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.489867 13967 raft_consensus.cc:2272] T 9d8fe0d92f6f47dfb304a24bf915e2db P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.491746 13967 tablet_replica.cc:333] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.492347 13967 raft_consensus.cc:2243] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.492919 13967 raft_consensus.cc:2272] T 09eea6ae09f04b0b9937f09218fcb3b1 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.494597 13967 tablet_replica.cc:333] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.495142 13967 raft_consensus.cc:2243] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.496019 13967 raft_consensus.cc:2272] T 07a1de4f2ab24624aa87e5619961710d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.498309 13967 tablet_replica.cc:333] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.499193 13967 raft_consensus.cc:2243] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.500247 13967 raft_consensus.cc:2272] T 15cd34bca3be499d87220d6e1b2a763f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.502326 13967 tablet_replica.cc:333] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.502874 13967 raft_consensus.cc:2243] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.503840 13967 raft_consensus.cc:2272] T 2fd68ef635784c42a7cc0ce9f2fae0f5 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.505854 13967 tablet_replica.cc:333] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.506448 13967 raft_consensus.cc:2243] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.507745 13967 raft_consensus.cc:2272] T ff3342a5b2ad49e2b252d7cd47b9851d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.510720 13967 tablet_replica.cc:333] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.511305 13967 raft_consensus.cc:2243] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.512151 13967 raft_consensus.cc:2272] T efef46e0b3c14af2a7179beba5c160b4 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.514321 13967 tablet_replica.cc:333] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.514896 13967 raft_consensus.cc:2243] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.515456 13967 raft_consensus.cc:2272] T 5e777c555e1b4153913b5e6459164454 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.517417 13967 tablet_replica.cc:333] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.517962 13967 raft_consensus.cc:2243] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.518419 13967 raft_consensus.cc:2272] T 6b6a82b88a6f4f6093256a0d85708d37 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.520262 13967 tablet_replica.cc:333] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.520816 13967 raft_consensus.cc:2243] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.521399 13967 raft_consensus.cc:2272] T 0a5f358b91ee427880ebecaca8118e4f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.523319 13967 tablet_replica.cc:333] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.523851 13967 raft_consensus.cc:2243] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.524714 13967 raft_consensus.cc:2272] T 365e6a51b3034d0483fdd8fc4cd362ea P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.526978 13967 tablet_replica.cc:333] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.527536 13967 raft_consensus.cc:2243] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.528460 13967 raft_consensus.cc:2272] T 16dab822009a4f00a0a480a48136ae25 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.530472 13967 tablet_replica.cc:333] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.531024 13967 raft_consensus.cc:2243] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.531514 13967 raft_consensus.cc:2272] T b358d067cea0428190a484601dea7c69 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.533489 13967 tablet_replica.cc:333] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.533998 13967 raft_consensus.cc:2243] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.534888 13967 raft_consensus.cc:2272] T 7defae86b18344bdb8b04e0d7c8f1a8d P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.536996 13967 tablet_replica.cc:333] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.537528 13967 raft_consensus.cc:2243] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.538417 13967 raft_consensus.cc:2272] T 14bd17b479354abca0014ae059714e8f P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.540432 13967 tablet_replica.cc:333] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.541023 13967 raft_consensus.cc:2243] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:16.541841 13967 raft_consensus.cc:2272] T 7f14609939d04f1e90837cb17fe19457 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:16.543812 13967 tablet_replica.cc:333] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82: stopping tablet replica
I20260709 13:06:16.544245 13967 raft_consensus.cc:2243] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:16.544719 13967 raft_consensus.cc:2272] T 42a84a565612402dbb559ec973692018 P 3c28be6f86ea4fb6b79a47d3999f5a82 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.059917 14975 maintenance_manager.cc:419] P df53447e208f455db54704b341851d53: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033400
I20260709 13:06:17.098268 14974 maintenance_manager.cc:643] P df53447e208f455db54704b341851d53: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s	user 0.031s	sys 0.005s Metrics: {"bytes_written":7572,"delete_count":0,"lbm_write_time_us":941,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":611,"threads_started":1,"update_count":298,"wal-append.queue_time_us":997}
I20260709 13:06:17.099807 14975 maintenance_manager.cc:419] P df53447e208f455db54704b341851d53: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.328831
I20260709 13:06:17.146842 13967 tablet_server.cc:196] TabletServer@127.13.163.196:0 shutdown complete.
I20260709 13:06:17.184796 14974 maintenance_manager.cc:643] P df53447e208f455db54704b341851d53: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.084s	user 0.083s	sys 0.001s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":28041,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":1848,"dirs.run_cpu_time_us":430,"dirs.run_wall_time_us":431,"lbm_read_time_us":1607,"lbm_reads_lt_1ms":24,"lbm_write_time_us":1391,"lbm_writes_lt_1ms":12,"peak_mem_usage":307519,"reinsert_count":0,"thread_start_us":1379,"threads_started":2,"update_count":298}
I20260709 13:06:17.225885 13967 master.cc:562] Master@127.13.163.254:45133 shutting down...
W20260709 13:06:40.257773 14970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111) [suppressed 660 similar messages]
W20260709 13:06:40.261384 14970 catalog_manager.cc:4755] TS 4b633b84351f40de9437e3d532fe6fa7 (127.13.163.195:42229): ChangeConfig:REMOVE_PEER RPC failed for tablet 301fa97ba426402092eaf85685303b33: Network error: Client connection negotiation failed: client connection to 127.13.163.195:42229: connect: Connection refused (error 111)
I20260709 13:06:43.817195 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.818406 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.818768 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P df53447e208f455db54704b341851d53: stopping tablet replica
I20260709 13:06:43.967012 13967 master.cc:584] Master@127.13.163.254:45133 shutdown complete.
I20260709 13:06:44.022015 13967 test_util.cc:182] -----------------------------------------------
I20260709 13:06:44.022243 13967 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602092437201-13967-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (284955 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:06:44.025868 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:39475
I20260709 13:06:44.027025 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:44.033143 23243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.033617 23244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.034080 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.034565 13967 server_base.cc:1061] running on GCE node
I20260709 13:06:44.038146 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:44.038340 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:44.038453 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602404038441 us; error 0 us; skew 500 ppm
I20260709 13:06:44.038887 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:44.041584 13967 webserver.cc:533] Webserver started at http://127.13.163.254:45075/ using document root <none> and password file <none>
I20260709 13:06:44.042044 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:44.042191 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:44.042402 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:44.043602 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "3a5a5c29c885437f816972f4b29bdccf"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.047991 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:44.051270 23251 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.052127 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:06:44.052455 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "3a5a5c29c885437f816972f4b29bdccf"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.052750 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:44.069969 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:44.071246 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:44.110433 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:39475
I20260709 13:06:44.110546 23302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:39475 every 8 connection(s)
I20260709 13:06:44.114733 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.126345 23303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: Bootstrap starting.
I20260709 13:06:44.131491 23303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.135807 23303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: No bootstrap required, opened a new log
I20260709 13:06:44.138202 23303 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER }
I20260709 13:06:44.138711 23303 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:44.138949 23303 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5a5c29c885437f816972f4b29bdccf, State: Initialized, Role: FOLLOWER
I20260709 13:06:44.139626 23303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER }
I20260709 13:06:44.140108 23303 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:44.140338 23303 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:44.140632 23303 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:44.145260 23303 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER }
I20260709 13:06:44.145807 23303 leader_election.cc:304] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a5a5c29c885437f816972f4b29bdccf; no voters: 
I20260709 13:06:44.146914 23303 leader_election.cc:290] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:44.147470 23306 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:44.148823 23306 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 1 LEADER]: Becoming Leader. State: Replica: 3a5a5c29c885437f816972f4b29bdccf, State: Running, Role: LEADER
I20260709 13:06:44.149572 23306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER }
I20260709 13:06:44.149969 23303 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:44.156562 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a5a5c29c885437f816972f4b29bdccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER } }
I20260709 13:06:44.156744 23308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a5a5c29c885437f816972f4b29bdccf. Latest consensus state: current_term: 1 leader_uuid: "3a5a5c29c885437f816972f4b29bdccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5a5c29c885437f816972f4b29bdccf" member_type: VOTER } }
I20260709 13:06:44.157287 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:44.157416 23308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:44.159413 23315 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:44.164244 23315 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:44.166657 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:44.173290 23315 catalog_manager.cc:1383] Generated new cluster ID: cab0d929fd4240b2bea02bdd1bde92b3
I20260709 13:06:44.173583 23315 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:44.187592 23315 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:44.188895 23315 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:44.201622 23315 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: Generated new TSK 0
I20260709 13:06:44.202260 23315 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:44.233001 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:44.239367 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.240451 23325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.241734 23327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.242712 13967 server_base.cc:1061] running on GCE node
I20260709 13:06:44.245250 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:44.245534 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:44.245708 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602404245675 us; error 0 us; skew 500 ppm
I20260709 13:06:44.246291 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:44.249166 13967 webserver.cc:533] Webserver started at http://127.13.163.193:34243/ using document root <none> and password file <none>
I20260709 13:06:44.249706 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:44.249869 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:44.250191 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:44.251575 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.257208 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:06:44.261091 23332 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.261946 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:44.262265 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.262552 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:44.281402 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:44.282617 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:44.284178 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:44.286691 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:44.286918 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.287171 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:44.287350 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.331574 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:35013
I20260709 13:06:44.331676 23394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:35013 every 8 connection(s)
I20260709 13:06:44.337082 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:44.345096 23399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.345484 23400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.349582 13967 server_base.cc:1061] running on GCE node
W20260709 13:06:44.350468 23402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.352942 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:44.353202 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:44.353391 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602404353370 us; error 0 us; skew 500 ppm
I20260709 13:06:44.353956 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:44.356369 23395 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39475
I20260709 13:06:44.356773 23395 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:44.357683 23395 heartbeater.cc:507] Master 127.13.163.254:39475 requested a full tablet report, sending...
I20260709 13:06:44.358045 13967 webserver.cc:533] Webserver started at http://127.13.163.194:37211/ using document root <none> and password file <none>
I20260709 13:06:44.358603 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:44.358790 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:44.359073 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:44.359903 23268 ts_manager.cc:194] Registered new tserver with Master: d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:44.360481 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.362041 23268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60476
I20260709 13:06:44.365506 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:44.369342 23407 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.370121 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:06:44.370458 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.370755 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:44.385053 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:44.386385 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:44.387965 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:44.390419 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:44.390695 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.390938 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:44.391103 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.434943 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:37247
I20260709 13:06:44.435068 23469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:37247 every 8 connection(s)
I20260709 13:06:44.440155 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:44.448055 23473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.450027 23474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.451654 13967 server_base.cc:1061] running on GCE node
W20260709 13:06:44.453267 23476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.453922 23470 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39475
I20260709 13:06:44.454298 23470 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:44.454361 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:44.454718 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:44.454963 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602404454939 us; error 0 us; skew 500 ppm
I20260709 13:06:44.455291 23470 heartbeater.cc:507] Master 127.13.163.254:39475 requested a full tablet report, sending...
I20260709 13:06:44.455641 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:44.457715 23268 ts_manager.cc:194] Registered new tserver with Master: 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:44.458737 13967 webserver.cc:533] Webserver started at http://127.13.163.195:37367/ using document root <none> and password file <none>
I20260709 13:06:44.459403 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:44.459637 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:44.459657 23268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60490
I20260709 13:06:44.460090 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:44.461982 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "12589c3399b641fc80f39058a475f8a1"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.466748 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:44.470112 23481 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.470873 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:44.471184 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "12589c3399b641fc80f39058a475f8a1"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.471418 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:44.486152 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:44.487370 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:44.488802 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:44.491189 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:44.491374 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.491698 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:44.491837 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.535940 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:36741
I20260709 13:06:44.536065 23543 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:36741 every 8 connection(s)
I20260709 13:06:44.542130 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:44.551631 23547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:44.551879 23548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.554785 23544 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39475
I20260709 13:06:44.555347 23544 heartbeater.cc:461] Registering TS with master...
W20260709 13:06:44.556450 23550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:44.556493 13967 server_base.cc:1061] running on GCE node
I20260709 13:06:44.556406 23544 heartbeater.cc:507] Master 127.13.163.254:39475 requested a full tablet report, sending...
I20260709 13:06:44.558090 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:44.558389 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:44.558650 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602404558628 us; error 0 us; skew 500 ppm
I20260709 13:06:44.559271 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:44.559583 23268 ts_manager.cc:194] Registered new tserver with Master: 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:44.561422 23268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60494
I20260709 13:06:44.562225 13967 webserver.cc:533] Webserver started at http://127.13.163.196:44509/ using document root <none> and password file <none>
I20260709 13:06:44.562897 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:44.563089 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:44.563359 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:44.564600 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-3-root/instance:
uuid: "ceb56da8fd5145648a898f5487219c6e"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.569268 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:44.573065 23555 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.573840 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:06:44.574215 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
uuid: "ceb56da8fd5145648a898f5487219c6e"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:44.574520 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:44.596359 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:44.597743 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:44.599205 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:44.601490 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:44.601715 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.602008 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:44.602193 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:44.644369 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.196:41791
I20260709 13:06:44.644479 23617 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.196:41791 every 8 connection(s)
I20260709 13:06:44.659852 23618 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39475
I20260709 13:06:44.660300 23618 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:44.661299 23618 heartbeater.cc:507] Master 127.13.163.254:39475 requested a full tablet report, sending...
I20260709 13:06:44.663206 23268 ts_manager.cc:194] Registered new tserver with Master: ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:44.663589 13967 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015786279s
I20260709 13:06:44.665048 23268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60496
I20260709 13:06:44.690790 23268 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60504:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:06:44.932245 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3cf23d64f04f4cb98bbd43988743ea (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:44.933487 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 77c7e59145ae42669dbe05a9943aa20e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:44.934057 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3cf23d64f04f4cb98bbd43988743ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.935263 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c7e59145ae42669dbe05a9943aa20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.938530 23508 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa7940429cf448f96c400f792bb6580 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:44.940347 23508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa7940429cf448f96c400f792bb6580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.940400 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3cf23d64f04f4cb98bbd43988743ea (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:44.941957 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3cf23d64f04f4cb98bbd43988743ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.945938 23583 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3cf23d64f04f4cb98bbd43988743ea (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:44.946440 23359 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa7940429cf448f96c400f792bb6580 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:44.947765 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3cf23d64f04f4cb98bbd43988743ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.947944 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa7940429cf448f96c400f792bb6580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.950157 23434 tablet_service.cc:1511] Processing CreateTablet for tablet d1fb77caf9eb4699b2319185288758e0 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:44.950332 23582 tablet_service.cc:1511] Processing CreateTablet for tablet d1fb77caf9eb4699b2319185288758e0 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:44.951679 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fb77caf9eb4699b2319185288758e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.951974 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fb77caf9eb4699b2319185288758e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.952879 23358 tablet_service.cc:1511] Processing CreateTablet for tablet d1fb77caf9eb4699b2319185288758e0 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:44.954015 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 77c7e59145ae42669dbe05a9943aa20e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:44.954245 23581 tablet_service.cc:1511] Processing CreateTablet for tablet 77c7e59145ae42669dbe05a9943aa20e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:44.955437 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c7e59145ae42669dbe05a9943aa20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.958125 23432 tablet_service.cc:1511] Processing CreateTablet for tablet 546616724fd047fdb579028e4581131e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:44.954290 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fb77caf9eb4699b2319185288758e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.959570 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546616724fd047fdb579028e4581131e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.961611 23435 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa7940429cf448f96c400f792bb6580 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:44.962751 23581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c7e59145ae42669dbe05a9943aa20e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.962965 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa7940429cf448f96c400f792bb6580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.978464 23638 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:44.982910 23639 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:44.985940 23638 tablet_bootstrap.cc:654] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:44.990393 23359 tablet_service.cc:1511] Processing CreateTablet for tablet 546616724fd047fdb579028e4581131e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:44.991883 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546616724fd047fdb579028e4581131e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:44.992616 23639 tablet_bootstrap.cc:654] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.005610 23639 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:45.008598 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 546616724fd047fdb579028e4581131e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:45.010222 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 546616724fd047fdb579028e4581131e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.014932 23638 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:45.015655 23638 ts_tablet_manager.cc:1403] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.000s
I20260709 13:06:45.015409 23580 tablet_service.cc:1511] Processing CreateTablet for tablet e1e2b1695b1849db90d73f0c6719c9d3 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:45.017159 23580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e2b1695b1849db90d73f0c6719c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.019253 23638 raft_consensus.cc:359] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.020081 23638 raft_consensus.cc:385] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.020403 23638 raft_consensus.cc:740] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.006306 23639 ts_tablet_manager.cc:1403] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 13:06:45.024649 23639 raft_consensus.cc:359] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.025671 23639 raft_consensus.cc:385] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.026077 23639 raft_consensus.cc:740] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.029210 23639 consensus_queue.cc:260] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.030092 23638 consensus_queue.cc:260] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.034734 23642 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:45.037623 23544 heartbeater.cc:499] Master 127.13.163.254:39475 was elected leader, sending a full tablet report...
I20260709 13:06:45.039077 23357 tablet_service.cc:1511] Processing CreateTablet for tablet e1e2b1695b1849db90d73f0c6719c9d3 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:45.050093 23579 tablet_service.cc:1511] Processing CreateTablet for tablet 1c16a9c7d17249d98740f56a32c03d84 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:45.051000 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e2b1695b1849db90d73f0c6719c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.051839 23579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c16a9c7d17249d98740f56a32c03d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.040984 23431 tablet_service.cc:1511] Processing CreateTablet for tablet e1e2b1695b1849db90d73f0c6719c9d3 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:45.054154 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e2b1695b1849db90d73f0c6719c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.060081 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 8678c528777f43c7b3552d5d8f484677 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:45.064951 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8678c528777f43c7b3552d5d8f484677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.060346 23506 tablet_service.cc:1511] Processing CreateTablet for tablet 8678c528777f43c7b3552d5d8f484677 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:45.066488 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8678c528777f43c7b3552d5d8f484677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.042997 23638 ts_tablet_manager.cc:1434] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:06:45.044142 23395 heartbeater.cc:499] Master 127.13.163.254:39475 was elected leader, sending a full tablet report...
I20260709 13:06:45.062340 23359 tablet_service.cc:1511] Processing CreateTablet for tablet 1c16a9c7d17249d98740f56a32c03d84 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:45.069752 23638 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:45.044226 23639 ts_tablet_manager.cc:1434] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:06:45.071050 23639 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:45.045929 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 1c16a9c7d17249d98740f56a32c03d84 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:45.081187 23639 tablet_bootstrap.cc:654] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.081612 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c16a9c7d17249d98740f56a32c03d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.090108 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c16a9c7d17249d98740f56a32c03d84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:45.091084 23396 tablet.cc:2406] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:45.076702 23638 tablet_bootstrap.cc:654] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.111914 23578 tablet_service.cc:1511] Processing CreateTablet for tablet 8678c528777f43c7b3552d5d8f484677 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:45.113476 23578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8678c528777f43c7b3552d5d8f484677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.121435 23642 tablet_bootstrap.cc:654] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.126436 23638 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:45.126613 23645 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:45.127130 23638 ts_tablet_manager.cc:1403] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.000s
I20260709 13:06:45.135937 23639 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:45.127821 23357 tablet_service.cc:1511] Processing CreateTablet for tablet c8912a0f40244d6bb5ff749263ff4993 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:45.137168 23645 tablet_bootstrap.cc:654] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.137362 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8912a0f40244d6bb5ff749263ff4993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.140324 23431 tablet_service.cc:1511] Processing CreateTablet for tablet c8912a0f40244d6bb5ff749263ff4993 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:45.136729 23639 ts_tablet_manager.cc:1403] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.066s	user 0.005s	sys 0.005s
I20260709 13:06:45.141674 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8912a0f40244d6bb5ff749263ff4993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.142788 23506 tablet_service.cc:1511] Processing CreateTablet for tablet c8912a0f40244d6bb5ff749263ff4993 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:45.144395 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8912a0f40244d6bb5ff749263ff4993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.148360 23578 tablet_service.cc:1511] Processing CreateTablet for tablet 70d9e995f5e94779b4b8d0a4f921a35c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:45.149729 23578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d9e995f5e94779b4b8d0a4f921a35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.152595 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 70d9e995f5e94779b4b8d0a4f921a35c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:45.144667 23639 raft_consensus.cc:359] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.154034 23639 raft_consensus.cc:385] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.154418 23639 raft_consensus.cc:740] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.156587 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 893c1c5680b947488f9d810e532fe7e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:45.157967 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 893c1c5680b947488f9d810e532fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.158645 23639 consensus_queue.cc:260] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.161811 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 70d9e995f5e94779b4b8d0a4f921a35c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
W20260709 13:06:45.162293 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:45.162936 23642 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:45.163173 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d9e995f5e94779b4b8d0a4f921a35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.163445 23642 ts_tablet_manager.cc:1403] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.129s	user 0.006s	sys 0.007s
I20260709 13:06:45.164505 23639 ts_tablet_manager.cc:1434] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.006s
I20260709 13:06:45.136093 23638 raft_consensus.cc:359] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.165678 23638 raft_consensus.cc:385] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.166132 23638 raft_consensus.cc:740] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.165789 23639 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:45.167603 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 893c1c5680b947488f9d810e532fe7e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:45.154055 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d9e995f5e94779b4b8d0a4f921a35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.171054 23642 raft_consensus.cc:359] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.171849 23642 raft_consensus.cc:385] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.172204 23642 raft_consensus.cc:740] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.171934 23579 tablet_service.cc:1511] Processing CreateTablet for tablet 893c1c5680b947488f9d810e532fe7e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:45.173403 23645 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:45.173110 23638 consensus_queue.cc:260] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.173979 23645 ts_tablet_manager.cc:1403] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.005s
I20260709 13:06:45.174528 23583 tablet_service.cc:1511] Processing CreateTablet for tablet cc86fab625684e049569b12b52fdac33 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:45.175424 23638 ts_tablet_manager.cc:1434] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.000s
I20260709 13:06:45.176450 23638 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:45.173064 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 893c1c5680b947488f9d810e532fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.178411 23432 tablet_service.cc:1511] Processing CreateTablet for tablet cc86fab625684e049569b12b52fdac33 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:45.181793 23579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 893c1c5680b947488f9d810e532fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.185165 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc86fab625684e049569b12b52fdac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.183163 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc86fab625684e049569b12b52fdac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.189220 23506 tablet_service.cc:1511] Processing CreateTablet for tablet cc86fab625684e049569b12b52fdac33 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:45.190730 23645 raft_consensus.cc:359] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:45.191527 23645 raft_consensus.cc:385] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.191813 23645 raft_consensus.cc:740] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.194325 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc86fab625684e049569b12b52fdac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.192694 23645 consensus_queue.cc:260] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:45.215833 23639 tablet_bootstrap.cc:654] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.204071 23642 consensus_queue.cc:260] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.218010 23359 tablet_service.cc:1511] Processing CreateTablet for tablet 31308be47968488d8fe1db62fc5e9d48 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:45.222539 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31308be47968488d8fe1db62fc5e9d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.223201 23645 ts_tablet_manager.cc:1434] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 13:06:45.223963 23618 heartbeater.cc:499] Master 127.13.163.254:39475 was elected leader, sending a full tablet report...
I20260709 13:06:45.224318 23645 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:45.225037 23434 tablet_service.cc:1511] Processing CreateTablet for tablet 31308be47968488d8fe1db62fc5e9d48 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:45.226375 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31308be47968488d8fe1db62fc5e9d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.246506 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 31308be47968488d8fe1db62fc5e9d48 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:45.247951 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31308be47968488d8fe1db62fc5e9d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.256458 23357 tablet_service.cc:1511] Processing CreateTablet for tablet 0108a896781443dbaf9caef1e04e5cfa (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:45.262254 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 0108a896781443dbaf9caef1e04e5cfa (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:45.266701 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0108a896781443dbaf9caef1e04e5cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.270068 23645 tablet_bootstrap.cc:654] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.264782 23579 tablet_service.cc:1511] Processing CreateTablet for tablet 0108a896781443dbaf9caef1e04e5cfa (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:45.274019 23579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0108a896781443dbaf9caef1e04e5cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.274799 23638 tablet_bootstrap.cc:654] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.276707 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0108a896781443dbaf9caef1e04e5cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.279659 23644 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.280566 23644 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.283270 23644 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:45.283571 23506 tablet_service.cc:1511] Processing CreateTablet for tablet 9af1e469bb524d288ab7911185a018b2 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:45.290438 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af1e469bb524d288ab7911185a018b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.292392 23650 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.293129 23650 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:45.295450 23583 tablet_service.cc:1511] Processing CreateTablet for tablet 9af1e469bb524d288ab7911185a018b2 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:45.305550 23650 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:45.310262 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af1e469bb524d288ab7911185a018b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.318178 23431 tablet_service.cc:1511] Processing CreateTablet for tablet fc4a932d15eb4e9e89e5b9bf6627de2b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:45.319669 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4a932d15eb4e9e89e5b9bf6627de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.329293 23509 tablet_service.cc:1511] Processing CreateTablet for tablet fc4a932d15eb4e9e89e5b9bf6627de2b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:45.330729 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4a932d15eb4e9e89e5b9bf6627de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.345034 23470 heartbeater.cc:499] Master 127.13.163.254:39475 was elected leader, sending a full tablet report...
I20260709 13:06:45.352520 23643 raft_consensus.cc:493] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.340013 23644 raft_consensus.cc:493] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.353194 23643 raft_consensus.cc:515] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.353319 23358 tablet_service.cc:1511] Processing CreateTablet for tablet aa6c9e6d5c9540229369e18bc85a6a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:45.359036 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6c9e6d5c9540229369e18bc85a6a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.359220 23639 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:45.359824 23639 ts_tablet_manager.cc:1403] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.194s	user 0.025s	sys 0.022s
I20260709 13:06:45.357558 23644 raft_consensus.cc:515] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.360252 23643 leader_election.cc:290] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:45.360533 23508 tablet_service.cc:1511] Processing CreateTablet for tablet aa6c9e6d5c9540229369e18bc85a6a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:45.364367 23434 tablet_service.cc:1511] Processing CreateTablet for tablet aa6c9e6d5c9540229369e18bc85a6a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:45.364966 23508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6c9e6d5c9540229369e18bc85a6a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.364665 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 9af1e469bb524d288ab7911185a018b2 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:45.365926 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6c9e6d5c9540229369e18bc85a6a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.366039 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af1e469bb524d288ab7911185a018b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.361830 23644 leader_election.cc:290] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:45.375113 23582 tablet_service.cc:1511] Processing CreateTablet for tablet 8118adb799df4bc2adcd22faba9e44d6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:45.376703 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8118adb799df4bc2adcd22faba9e44d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.390506 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 8118adb799df4bc2adcd22faba9e44d6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:45.392006 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8118adb799df4bc2adcd22faba9e44d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.362716 23643 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.352376 23578 tablet_service.cc:1511] Processing CreateTablet for tablet fc4a932d15eb4e9e89e5b9bf6627de2b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:45.392853 23642 ts_tablet_manager.cc:1434] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.229s	user 0.020s	sys 0.013s
I20260709 13:06:45.412794 23645 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:45.415131 23642 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:45.415788 23578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4a932d15eb4e9e89e5b9bf6627de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.417294 23645 ts_tablet_manager.cc:1403] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.193s	user 0.020s	sys 0.009s
I20260709 13:06:45.424762 23645 raft_consensus.cc:359] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.400434 23643 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.429265 23645 raft_consensus.cc:385] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.426435 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:45.440281 23638 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
W20260709 13:06:45.441974 23558 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.451462 23638 ts_tablet_manager.cc:1403] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.275s	user 0.001s	sys 0.029s
I20260709 13:06:45.469123 23432 tablet_service.cc:1511] Processing CreateTablet for tablet 56c0cd2ce17e48709b4a82c121539b62 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:45.469146 23581 tablet_service.cc:1511] Processing CreateTablet for tablet be3f330957084b66847db5f71ee7bc36 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:45.470533 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c0cd2ce17e48709b4a82c121539b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.470216 23357 tablet_service.cc:1511] Processing CreateTablet for tablet be3f330957084b66847db5f71ee7bc36 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:45.470646 23581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be3f330957084b66847db5f71ee7bc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.471227 23645 raft_consensus.cc:740] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.471782 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be3f330957084b66847db5f71ee7bc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.475132 23642 tablet_bootstrap.cc:654] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.475800 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 8118adb799df4bc2adcd22faba9e44d6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:45.477453 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8118adb799df4bc2adcd22faba9e44d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.490499 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:45.491325 23519 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:45.492339 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:45.492682 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:45.493791 23651 raft_consensus.cc:493] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.494354 23651 raft_consensus.cc:515] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:45.495373 23556 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.495875 23556 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:45.497109 23650 raft_consensus.cc:2749] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:45.508350 23333 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:45.515035 23644 raft_consensus.cc:2804] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:45.515555 23644 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.515897 23644 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.465233 23638 raft_consensus.cc:359] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.537765 23638 raft_consensus.cc:385] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.538201 23638 raft_consensus.cc:740] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.539099 23638 consensus_queue.cc:260] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.550325 23639 raft_consensus.cc:359] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.584934 23639 raft_consensus.cc:385] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.585315 23639 raft_consensus.cc:740] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.586129 23639 consensus_queue.cc:260] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.588481 23639 ts_tablet_manager.cc:1434] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.228s	user 0.007s	sys 0.007s
I20260709 13:06:45.550675 23645 consensus_queue.cc:260] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.590499 23645 ts_tablet_manager.cc:1434] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.169s	user 0.011s	sys 0.001s
I20260709 13:06:45.591487 23645 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:45.593915 23638 ts_tablet_manager.cc:1434] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.142s	user 0.009s	sys 0.000s
I20260709 13:06:45.595116 23638 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:45.618079 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:45.564447 23642 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
W20260709 13:06:45.566202 23252 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 68237.2us
I20260709 13:06:45.619841 23642 ts_tablet_manager.cc:1403] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.205s	user 0.014s	sys 0.003s
I20260709 13:06:45.569803 23651 leader_election.cc:290] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
W20260709 13:06:45.571362 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 68915.2us
W20260709 13:06:45.623210 23333 leader_election.cc:343] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.623636 23583 tablet_service.cc:1511] Processing CreateTablet for tablet 56c0cd2ce17e48709b4a82c121539b62 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:45.625000 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c0cd2ce17e48709b4a82c121539b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.625401 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 56c0cd2ce17e48709b4a82c121539b62 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:45.625849 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:45.626801 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c0cd2ce17e48709b4a82c121539b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.628074 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:45.628746 23443 raft_consensus.cc:2468] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:45.623221 23642 raft_consensus.cc:359] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.577399 23643 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:45.599009 23639 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:45.631094 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:45.631814 23593 raft_consensus.cc:2468] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:45.633401 23482 leader_election.cc:304] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
W20260709 13:06:45.637189 23333 leader_election.cc:343] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.637570 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 02d63cb871f04825b6880a4e77c0f97b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:45.638887 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d63cb871f04825b6880a4e77c0f97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.638398 23333 leader_election.cc:304] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:45.639878 23669 raft_consensus.cc:2804] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:45.640247 23669 raft_consensus.cc:493] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.638934 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:45.640581 23669 raft_consensus.cc:3060] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.641355 23518 raft_consensus.cc:2468] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:45.642818 23408 leader_election.cc:304] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
W20260709 13:06:45.559748 23254 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 90267.3us
I20260709 13:06:45.608146 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:45.650301 23651 raft_consensus.cc:2804] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:45.603904 23645 tablet_bootstrap.cc:654] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.650744 23651 raft_consensus.cc:493] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.651152 23651 raft_consensus.cc:3060] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.608260 23506 tablet_service.cc:1511] Processing CreateTablet for tablet be3f330957084b66847db5f71ee7bc36 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:45.609750 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:45.629635 23642 raft_consensus.cc:385] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.653936 23642 raft_consensus.cc:740] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.654758 23642 consensus_queue.cc:260] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.638820 23643 raft_consensus.cc:2804] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:45.656023 23643 raft_consensus.cc:493] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.656419 23643 raft_consensus.cc:3060] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.656112 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 02d63cb871f04825b6880a4e77c0f97b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:45.657581 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d63cb871f04825b6880a4e77c0f97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.662411 23357 tablet_service.cc:1511] Processing CreateTablet for tablet c4bb4f04978d43798dc3c00c13d510be (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:45.663784 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4bb4f04978d43798dc3c00c13d510be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.663571 23356 tablet_service.cc:1511] Processing CreateTablet for tablet f954a7fd7b3840a9b646d16a33cab480 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:45.665035 23356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f954a7fd7b3840a9b646d16a33cab480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.665784 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 538d207f493d4b14b97bce7658551672 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:45.667057 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538d207f493d4b14b97bce7658551672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.669126 23642 ts_tablet_manager.cc:1434] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:06:45.669294 23359 tablet_service.cc:1511] Processing CreateTablet for tablet 37ece810899b40429ea55aa6a3642a63 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:45.670159 23642 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:45.670595 23355 tablet_service.cc:1511] Processing CreateTablet for tablet ff658c154728474ab62925bcfa9e4fae (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:45.671962 23355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff658c154728474ab62925bcfa9e4fae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.673870 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be3f330957084b66847db5f71ee7bc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.676955 23639 tablet_bootstrap.cc:654] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.677177 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ece810899b40429ea55aa6a3642a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.608968 23644 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.682924 23370 raft_consensus.cc:2393] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
W20260709 13:06:45.689195 23482 leader_election.cc:343] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:45.690366 23482 leader_election.cc:343] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.690585 23583 tablet_service.cc:1511] Processing CreateTablet for tablet c4bb4f04978d43798dc3c00c13d510be (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:45.692078 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4bb4f04978d43798dc3c00c13d510be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.695845 23432 tablet_service.cc:1511] Processing CreateTablet for tablet c4bb4f04978d43798dc3c00c13d510be (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:45.697284 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4bb4f04978d43798dc3c00c13d510be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.697098 23578 tablet_service.cc:1511] Processing CreateTablet for tablet 538d207f493d4b14b97bce7658551672 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:45.698608 23578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538d207f493d4b14b97bce7658551672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.700719 23354 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa87a90cab0475da44d21f725d485ab (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:45.702332 23354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa87a90cab0475da44d21f725d485ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.702865 23431 tablet_service.cc:1511] Processing CreateTablet for tablet 7d1599ca0cb14ed39d2ffde42649f5e6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:45.704337 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1599ca0cb14ed39d2ffde42649f5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.705986 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 3a01f067a6214e979fafb367a095f1c1 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:45.707536 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a01f067a6214e979fafb367a095f1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.667629 23643 raft_consensus.cc:515] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.714963 23643 leader_election.cc:290] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:45.715270 23579 tablet_service.cc:1511] Processing CreateTablet for tablet 7d1599ca0cb14ed39d2ffde42649f5e6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:45.716827 23579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1599ca0cb14ed39d2ffde42649f5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.721714 23484 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:45.730005 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 02d63cb871f04825b6880a4e77c0f97b (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:45.731284 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:45.730077 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:45.731575 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d63cb871f04825b6880a4e77c0f97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.732419 23638 tablet_bootstrap.cc:654] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.732015 23593 raft_consensus.cc:3060] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:45.737003 23482 leader_election.cc:343] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.737437 23508 tablet_service.cc:1511] Processing CreateTablet for tablet 538d207f493d4b14b97bce7658551672 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:45.737664 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 7d1599ca0cb14ed39d2ffde42649f5e6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:45.739205 23508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538d207f493d4b14b97bce7658551672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.740481 23642 tablet_bootstrap.cc:654] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:45.741418 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1599ca0cb14ed39d2ffde42649f5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.746953 23669 raft_consensus.cc:515] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.745200 23643 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.759840 23645 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:45.760440 23645 ts_tablet_manager.cc:1403] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.169s	user 0.000s	sys 0.024s
I20260709 13:06:45.763231 23651 raft_consensus.cc:515] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.773715 23505 tablet_service.cc:1511] Processing CreateTablet for tablet 37ece810899b40429ea55aa6a3642a63 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:45.773926 23582 tablet_service.cc:1511] Processing CreateTablet for tablet f954a7fd7b3840a9b646d16a33cab480 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:45.775708 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:45.773999 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:45.775574 23580 tablet_service.cc:1511] Processing CreateTablet for tablet ff658c154728474ab62925bcfa9e4fae (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:45.782330 23370 raft_consensus.cc:2393] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:45.782999 23580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff658c154728474ab62925bcfa9e4fae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.783924 23432 tablet_service.cc:1511] Processing CreateTablet for tablet 37ece810899b40429ea55aa6a3642a63 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:45.784076 23503 tablet_service.cc:1511] Processing CreateTablet for tablet 21203c7d4b1148e1a4e659daf1df428c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:45.785363 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21203c7d4b1148e1a4e659daf1df428c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.788287 23644 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:45.776587 23504 tablet_service.cc:1511] Processing CreateTablet for tablet ff658c154728474ab62925bcfa9e4fae (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:45.788954 23593 raft_consensus.cc:2468] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:45.789093 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:45.790060 23504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff658c154728474ab62925bcfa9e4fae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:45.791028 23333 leader_election.cc:343] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:45.765455 23645 raft_consensus.cc:359] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.780776 23444 raft_consensus.cc:2393] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:45.781158 23435 tablet_service.cc:1511] Processing CreateTablet for tablet 21203c7d4b1148e1a4e659daf1df428c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:45.781423 23434 tablet_service.cc:1511] Processing CreateTablet for tablet f954a7fd7b3840a9b646d16a33cab480 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:45.793176 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:45.794446 23639 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:45.794973 23639 ts_tablet_manager.cc:1403] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.196s	user 0.017s	sys 0.006s
I20260709 13:06:45.796494 23645 raft_consensus.cc:385] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.796947 23645 raft_consensus.cc:740] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.797070 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f954a7fd7b3840a9b646d16a33cab480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.785331 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ece810899b40429ea55aa6a3642a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.798820 23505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ece810899b40429ea55aa6a3642a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.798426 23482 leader_election.cc:304] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:45.809043 23429 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa87a90cab0475da44d21f725d485ab (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:45.829561 23661 raft_consensus.cc:2804] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:45.809213 23577 tablet_service.cc:1511] Processing CreateTablet for tablet 21203c7d4b1148e1a4e659daf1df428c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:45.810108 23651 leader_election.cc:290] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:45.810812 23430 tablet_service.cc:1511] Processing CreateTablet for tablet 3a01f067a6214e979fafb367a095f1c1 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:45.833671 23577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21203c7d4b1148e1a4e659daf1df428c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.833765 23501 tablet_service.cc:1511] Processing CreateTablet for tablet d1dd65a066e84285a82713f6ee51aa7e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:45.835450 23501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1dd65a066e84285a82713f6ee51aa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.830215 23429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa87a90cab0475da44d21f725d485ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.838371 23502 tablet_service.cc:1511] Processing CreateTablet for tablet 3a01f067a6214e979fafb367a095f1c1 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:45.840009 23502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a01f067a6214e979fafb367a095f1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.811342 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:45.841171 23651 raft_consensus.cc:493] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.812575 23576 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa87a90cab0475da44d21f725d485ab (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:45.841778 23651 raft_consensus.cc:515] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.842772 23576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa87a90cab0475da44d21f725d485ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.813779 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:45.844743 23651 leader_election.cc:290] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:45.845484 23519 raft_consensus.cc:3060] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.808574 23639 raft_consensus.cc:359] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.849368 23639 raft_consensus.cc:385] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:45.849727 23639 raft_consensus.cc:740] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:45.850989 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fb77caf9eb4699b2319185288758e0" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:45.851985 23369 raft_consensus.cc:2468] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:45.841470 23430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a01f067a6214e979fafb367a095f1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.852254 23639 consensus_queue.cc:260] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.841637 23370 raft_consensus.cc:2393] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:45.861457 23410 leader_election.cc:304] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:45.876919 23661 raft_consensus.cc:697] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:45.878090 23661 consensus_queue.cc:237] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:45.879055 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21203c7d4b1148e1a4e659daf1df428c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.879171 23669 leader_election.cc:290] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:45.880479 23645 consensus_queue.cc:260] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:45.883750 23651 raft_consensus.cc:2804] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:45.880532 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa7940429cf448f96c400f792bb6580" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:45.902407 23354 tablet_service.cc:1511] Processing CreateTablet for tablet d1dd65a066e84285a82713f6ee51aa7e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:45.903767 23354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1dd65a066e84285a82713f6ee51aa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.906423 23519 raft_consensus.cc:2468] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:45.907891 23408 leader_election.cc:304] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:45.910583 23671 raft_consensus.cc:2804] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:45.899302 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f954a7fd7b3840a9b646d16a33cab480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.915688 23517 raft_consensus.cc:2393] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:45.917804 23581 tablet_service.cc:1511] Processing CreateTablet for tablet d1dd65a066e84285a82713f6ee51aa7e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:45.919220 23333 leader_election.cc:304] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:45.920506 23669 raft_consensus.cc:2749] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:45.899173 23651 raft_consensus.cc:493] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.927305 23581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1dd65a066e84285a82713f6ee51aa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.923640 23651 raft_consensus.cc:3060] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:45.931600 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fb77caf9eb4699b2319185288758e0" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:45.931496 23506 tablet_service.cc:1511] Processing CreateTablet for tablet 0664d2cef22d4b9fbcce815319e6e56c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:45.945533 23645 ts_tablet_manager.cc:1434] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.185s	user 0.005s	sys 0.007s
I20260709 13:06:45.946751 23645 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:45.939186 23638 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:45.952392 23575 tablet_service.cc:1511] Processing CreateTablet for tablet 0664d2cef22d4b9fbcce815319e6e56c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:45.953708 23593 raft_consensus.cc:2468] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:45.964391 23638 ts_tablet_manager.cc:1403] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.369s	user 0.014s	sys 0.005s
I20260709 13:06:45.959564 23671 raft_consensus.cc:697] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:45.967024 23575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0664d2cef22d4b9fbcce815319e6e56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.972289 23642 tablet_bootstrap.cc:492] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:45.975356 23642 ts_tablet_manager.cc:1403] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.305s	user 0.049s	sys 0.039s
I20260709 13:06:45.977109 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0664d2cef22d4b9fbcce815319e6e56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.986004 23433 tablet_service.cc:1511] Processing CreateTablet for tablet 671e4639fc5945aba1d87c67ed1fc249 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:45.987437 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 671e4639fc5945aba1d87c67ed1fc249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.991400 23643 raft_consensus.cc:2749] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:45.983163 23671 consensus_queue.cc:237] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:45.993434 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 671e4639fc5945aba1d87c67ed1fc249 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:45.998771 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 671e4639fc5945aba1d87c67ed1fc249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.999574 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2a928268b4d73afa32a47bbbcbd64 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:46.012519 23431 tablet_service.cc:1511] Processing CreateTablet for tablet 0664d2cef22d4b9fbcce815319e6e56c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:46.014554 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0664d2cef22d4b9fbcce815319e6e56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.991549 23642 raft_consensus.cc:359] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.028290 23642 raft_consensus.cc:385] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.028679 23642 raft_consensus.cc:740] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.031812 23642 consensus_queue.cc:260] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.034848 23508 tablet_service.cc:1511] Processing CreateTablet for tablet 671e4639fc5945aba1d87c67ed1fc249 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:46.036321 23508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 671e4639fc5945aba1d87c67ed1fc249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.987329 23638 raft_consensus.cc:359] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.045469 23638 raft_consensus.cc:385] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.046376 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2a928268b4d73afa32a47bbbcbd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.051651 23642 ts_tablet_manager.cc:1434] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.003s
I20260709 13:06:46.052196 23580 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2a928268b4d73afa32a47bbbcbd64 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:46.053629 23580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2a928268b4d73afa32a47bbbcbd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.053905 23645 tablet_bootstrap.cc:654] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.063668 23357 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1d7ecd2e941d289aefe4228c0e673 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:46.049661 23638 raft_consensus.cc:740] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.065204 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1d7ecd2e941d289aefe4228c0e673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.065588 23638 consensus_queue.cc:260] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.068941 23642 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:46.065634 23429 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2a928268b4d73afa32a47bbbcbd64 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:46.070916 23429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2a928268b4d73afa32a47bbbcbd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.074446 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1d7ecd2e941d289aefe4228c0e673 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:46.075265 23643 raft_consensus.cc:493] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.075495 23639 ts_tablet_manager.cc:1434] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.280s	user 0.030s	sys 0.008s
I20260709 13:06:46.075819 23643 raft_consensus.cc:515] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.076908 23639 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:46.079185 23643 leader_election.cc:290] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:46.080808 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:46.082417 23482 leader_election.cc:343] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:46.079453 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:46.083513 23370 raft_consensus.cc:2468] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:46.084893 23484 leader_election.cc:304] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:46.086316 23638 ts_tablet_manager.cc:1434] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.112s	user 0.009s	sys 0.001s
I20260709 13:06:46.087500 23638 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:46.088752 23643 raft_consensus.cc:2804] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:46.090415 23643 raft_consensus.cc:493] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.090767 23643 raft_consensus.cc:3060] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.091661 23428 tablet_service.cc:1511] Processing CreateTablet for tablet 90f7c7f831d14ec5870cb93e9b1f9f02 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:46.093425 23428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f7c7f831d14ec5870cb93e9b1f9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.099102 23503 tablet_service.cc:1511] Processing CreateTablet for tablet 90f7c7f831d14ec5870cb93e9b1f9f02 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:46.099480 23518 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:46.106271 23642 tablet_bootstrap.cc:654] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.112164 23638 tablet_bootstrap.cc:654] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.075850 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1d7ecd2e941d289aefe4228c0e673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.118093 23578 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1d7ecd2e941d289aefe4228c0e673 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:46.151829 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f7c7f831d14ec5870cb93e9b1f9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.168437 23578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1d7ecd2e941d289aefe4228c0e673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.137977 23333 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:46.184060 23669 raft_consensus.cc:2804] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:46.185636 23669 raft_consensus.cc:697] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:46.183666 23651 raft_consensus.cc:515] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.186460 23669 consensus_queue.cc:237] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:46.189137 23638 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:46.189605 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fb77caf9eb4699b2319185288758e0" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:46.192596 23593 raft_consensus.cc:3060] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.192691 23638 ts_tablet_manager.cc:1403] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.105s	user 0.036s	sys 0.013s
W20260709 13:06:46.194525 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:46.196192 23638 raft_consensus.cc:359] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:46.197052 23638 raft_consensus.cc:385] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.197412 23638 raft_consensus.cc:740] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.202939 23638 consensus_queue.cc:260] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:46.205385 23638 ts_tablet_manager.cc:1434] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:46.201957 23593 raft_consensus.cc:2468] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:46.206553 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fb77caf9eb4699b2319185288758e0" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:46.208201 23651 leader_election.cc:290] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:46.173378 23643 raft_consensus.cc:515] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.212705 23268 catalog_manager.cc:5697] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:46.215799 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 85896.1us
I20260709 13:06:46.219344 23501 tablet_service.cc:1511] Processing CreateTablet for tablet 44df4021944a47c9b442ffde13399f34 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:46.223299 23502 tablet_service.cc:1511] Processing CreateTablet for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:46.228947 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 4743addeb1314388b9469d11e0ff9385 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:46.225804 23434 tablet_service.cc:1511] Processing CreateTablet for tablet f7d59103647944ec805367e074fecea6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:46.233364 23433 tablet_service.cc:1511] Processing CreateTablet for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:46.233850 23575 tablet_service.cc:1511] Processing CreateTablet for tablet 4743addeb1314388b9469d11e0ff9385 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:46.228911 23432 tablet_service.cc:1511] Processing CreateTablet for tablet 4743addeb1314388b9469d11e0ff9385 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:46.237270 23435 tablet_service.cc:1511] Processing CreateTablet for tablet b57c381a872f4c7aae7b11e12cc439b8 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:46.238843 23576 tablet_service.cc:1511] Processing CreateTablet for tablet b57c381a872f4c7aae7b11e12cc439b8 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:46.241266 23266 catalog_manager.cc:5697] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:46.243710 23577 tablet_service.cc:1511] Processing CreateTablet for tablet f7d59103647944ec805367e074fecea6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:46.244139 23508 tablet_service.cc:1511] Processing CreateTablet for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:46.246060 23639 tablet_bootstrap.cc:654] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.245855 23581 tablet_service.cc:1511] Processing CreateTablet for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:46.249269 23370 raft_consensus.cc:3060] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.276347 23580 tablet_service.cc:1511] Processing CreateTablet for tablet 90f7c7f831d14ec5870cb93e9b1f9f02 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:46.278393 23508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.281051 23645 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:46.283293 23651 raft_consensus.cc:493] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.283222 23506 tablet_service.cc:1511] Processing CreateTablet for tablet 376a2904d9ff484195a63051a29b8a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:46.284418 23575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4743addeb1314388b9469d11e0ff9385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.289628 23357 tablet_service.cc:1511] Processing CreateTablet for tablet 44df4021944a47c9b442ffde13399f34 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:46.289912 23360 tablet_service.cc:1511] Processing CreateTablet for tablet f7d59103647944ec805367e074fecea6 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:46.290253 23354 tablet_service.cc:1511] Processing CreateTablet for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:46.292249 23355 tablet_service.cc:1511] Processing CreateTablet for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:46.296185 23638 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:46.303105 23359 tablet_service.cc:1511] Processing CreateTablet for tablet b57c381a872f4c7aae7b11e12cc439b8 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:46.304214 23370 raft_consensus.cc:2468] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:46.322122 23577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d59103647944ec805367e074fecea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.304462 23642 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:46.322885 23581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.323169 23642 ts_tablet_manager.cc:1403] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.271s	user 0.016s	sys 0.016s
I20260709 13:06:46.323531 23580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f7c7f831d14ec5870cb93e9b1f9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.305874 23643 leader_election.cc:290] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:46.307297 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:46.325176 23369 raft_consensus.cc:3060] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.309326 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:46.325941 23503 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8c839e30fa42ee9ff04b9ba759fc85 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:46.310492 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44df4021944a47c9b442ffde13399f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.310750 23429 tablet_service.cc:1511] Processing CreateTablet for tablet 44df4021944a47c9b442ffde13399f34 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:46.329723 23354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.330585 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57c381a872f4c7aae7b11e12cc439b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:46.331054 23482 leader_election.cc:343] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:46.311658 23267 catalog_manager.cc:5697] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:46.274526 23408 leader_election.cc:304] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
W20260709 13:06:46.332370 23408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 61321.2us
I20260709 13:06:46.334913 23583 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8c839e30fa42ee9ff04b9ba759fc85 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:46.313256 23502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5c2c534bbd4c909a5d8fd8e8ce52e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.296344 23433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.334954 23671 raft_consensus.cc:2804] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:46.337575 23671 raft_consensus.cc:697] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:46.338793 23671 consensus_queue.cc:237] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.297962 23651 raft_consensus.cc:515] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.317112 23582 tablet_service.cc:1511] Processing CreateTablet for tablet 376a2904d9ff484195a63051a29b8a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:46.341697 23429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44df4021944a47c9b442ffde13399f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.343166 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4743addeb1314388b9469d11e0ff9385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.343516 23506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376a2904d9ff484195a63051a29b8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.320515 23431 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8c839e30fa42ee9ff04b9ba759fc85 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:46.336639 23583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8c839e30fa42ee9ff04b9ba759fc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.346251 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376a2904d9ff484195a63051a29b8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.326114 23645 ts_tablet_manager.cc:1403] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.380s	user 0.037s	sys 0.025s
I20260709 13:06:46.331243 23355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7ef8cd1ea7446f6bd4a4f133cbf701c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.352299 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d59103647944ec805367e074fecea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.345439 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8c839e30fa42ee9ff04b9ba759fc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.354625 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57c381a872f4c7aae7b11e12cc439b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.356529 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d59103647944ec805367e074fecea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.359413 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4743addeb1314388b9469d11e0ff9385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.311718 23358 tablet_service.cc:1511] Processing CreateTablet for tablet 376a2904d9ff484195a63051a29b8a0d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:46.361379 23358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376a2904d9ff484195a63051a29b8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.364727 23645 raft_consensus.cc:359] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.365638 23645 raft_consensus.cc:385] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.367789 23645 raft_consensus.cc:740] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.367044 23356 tablet_service.cc:1511] Processing CreateTablet for tablet 18748afbb84547cba3ae58d7476a2119 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:46.369473 23428 tablet_service.cc:1511] Processing CreateTablet for tablet 18748afbb84547cba3ae58d7476a2119 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:46.405900 23356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18748afbb84547cba3ae58d7476a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.406139 23428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18748afbb84547cba3ae58d7476a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.408882 23430 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ce58a33542c395d1e4e41fe71ea7 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:46.410264 23430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ce58a33542c395d1e4e41fe71ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.412322 23369 raft_consensus.cc:2468] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:46.370375 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:46.413184 23501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44df4021944a47c9b442ffde13399f34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.413832 23484 leader_election.cc:304] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:46.415145 23643 raft_consensus.cc:2804] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:46.415679 23643 raft_consensus.cc:697] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:46.416091 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8c839e30fa42ee9ff04b9ba759fc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.416613 23643 consensus_queue.cc:237] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.420279 23638 tablet_bootstrap.cc:654] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.370577 23651 leader_election.cc:290] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:46.371181 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77c7e59145ae42669dbe05a9943aa20e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:46.421430 23593 raft_consensus.cc:2393] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:46.421231 23645 consensus_queue.cc:260] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.381207 23576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57c381a872f4c7aae7b11e12cc439b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.386099 23579 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ce58a33542c395d1e4e41fe71ea7 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:46.387507 23639 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:46.422897 23639 ts_tablet_manager.cc:1403] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.346s	user 0.013s	sys 0.011s
I20260709 13:06:46.424028 23579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ce58a33542c395d1e4e41fe71ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.400866 23353 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ce58a33542c395d1e4e41fe71ea7 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:46.379441 23505 tablet_service.cc:1511] Processing CreateTablet for tablet 18748afbb84547cba3ae58d7476a2119 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:46.424973 23408 leader_election.cc:304] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:46.425765 23353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ce58a33542c395d1e4e41fe71ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.426824 23677 raft_consensus.cc:3060] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.427661 23639 raft_consensus.cc:359] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.428622 23639 raft_consensus.cc:385] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.425781 23505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18748afbb84547cba3ae58d7476a2119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.429013 23639 raft_consensus.cc:740] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.430500 23639 consensus_queue.cc:260] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.366472 23642 raft_consensus.cc:359] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:46.458731 23639 ts_tablet_manager.cc:1434] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:06:46.460099 23639 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:46.444543 23642 raft_consensus.cc:385] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.461028 23642 raft_consensus.cc:740] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.462016 23642 consensus_queue.cc:260] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:46.464175 23642 ts_tablet_manager.cc:1434] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.141s	user 0.001s	sys 0.010s
I20260709 13:06:46.467803 23639 tablet_bootstrap.cc:654] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.477651 23651 consensus_queue.cc:1048] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:46.500615 23509 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3067bf3f594c4985b356c81a410f6c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:46.504141 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3067bf3f594c4985b356c81a410f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.505300 23644 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.505913 23644 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:46.508620 23644 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:46.502319 23677 raft_consensus.cc:2749] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:46.541857 23642 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:46.579909 23645 ts_tablet_manager.cc:1434] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.230s	user 0.007s	sys 0.015s
I20260709 13:06:46.581199 23645 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:46.596870 23582 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3067bf3f594c4985b356c81a410f6c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:46.612001 23690 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.617429 23690 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:46.621384 23435 tablet_service.cc:1511] Processing CreateTablet for tablet 3038aa8132294ba7832fad08934a848f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:46.622391 23690 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:46.622908 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038aa8132294ba7832fad08934a848f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.624689 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:46.627467 23687 log.cc:927] Time spent T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:46.629908 23428 tablet_service.cc:1511] Processing CreateTablet for tablet c20e8e3d7d4343c88e56c92382808e9d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:06:46.613150 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 57217.4us
I20260709 13:06:46.632018 23428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e8e3d7d4343c88e56c92382808e9d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:46.633504 23693 log.cc:927] Time spent T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:06:46.638383 23505 tablet_service.cc:1511] Processing CreateTablet for tablet 3038aa8132294ba7832fad08934a848f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:46.615504 23643 consensus_queue.cc:1048] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:06:46.640460 23661 consensus_queue.cc:1048] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:46.647857 23575 tablet_service.cc:1511] Processing CreateTablet for tablet 3038aa8132294ba7832fad08934a848f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 13:06:46.648896 23686 log.cc:927] Time spent T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:06:46.652592 23690 consensus_queue.cc:1048] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:46.653434 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3067bf3f594c4985b356c81a410f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.663625 23645 tablet_bootstrap.cc:654] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.670117 23638 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:46.670354 23639 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:46.670747 23638 ts_tablet_manager.cc:1403] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.399s	user 0.013s	sys 0.005s
I20260709 13:06:46.670841 23639 ts_tablet_manager.cc:1403] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.211s	user 0.025s	sys 0.030s
I20260709 13:06:46.675518 23642 tablet_bootstrap.cc:654] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.686960 23505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038aa8132294ba7832fad08934a848f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.693930 23639 raft_consensus.cc:359] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.694370 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:46.694777 23639 raft_consensus.cc:385] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.695134 23639 raft_consensus.cc:740] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.695212 23518 raft_consensus.cc:2393] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:46.709010 23408 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:46.718019 23690 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.718613 23502 tablet_service.cc:1511] Processing CreateTablet for tablet c20e8e3d7d4343c88e56c92382808e9d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:46.720026 23502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e8e3d7d4343c88e56c92382808e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.720242 23575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038aa8132294ba7832fad08934a848f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.739377 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:46.748540 23434 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf62d6f4e045f8b06e7170496fda73 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:46.750000 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf62d6f4e045f8b06e7170496fda73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.755105 23638 raft_consensus.cc:359] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.756018 23638 raft_consensus.cc:385] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.714262 23639 consensus_queue.cc:260] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.762169 23639 ts_tablet_manager.cc:1434] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.003s
I20260709 13:06:46.759742 23638 raft_consensus.cc:740] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.757990 23690 raft_consensus.cc:2749] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:46.763480 23639 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:46.790323 23576 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf62d6f4e045f8b06e7170496fda73 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:46.791780 23576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf62d6f4e045f8b06e7170496fda73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.820384 23661 raft_consensus.cc:493] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.821107 23661 raft_consensus.cc:515] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.823170 23661 leader_election.cc:290] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:46.820001 23577 tablet_service.cc:1511] Processing CreateTablet for tablet ec789761c5344074974b9c23b79e418a (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:46.826702 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:46.781237 23638 consensus_queue.cc:260] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.831511 23503 tablet_service.cc:1511] Processing CreateTablet for tablet ec789761c5344074974b9c23b79e418a (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:46.851120 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec789761c5344074974b9c23b79e418a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.832633 23577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec789761c5344074974b9c23b79e418a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.833828 23428 tablet_service.cc:1511] Processing CreateTablet for tablet 37d36e08707f4d8295154d67b1450b0f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:46.834671 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:46.841643 23354 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3067bf3f594c4985b356c81a410f6c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:46.841944 23355 tablet_service.cc:1511] Processing CreateTablet for tablet c20e8e3d7d4343c88e56c92382808e9d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:46.842736 23643 raft_consensus.cc:493] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.842556 23268 catalog_manager.cc:5697] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:46.846199 23645 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:46.845876 23353 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf62d6f4e045f8b06e7170496fda73 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:46.847939 23359 tablet_service.cc:1511] Processing CreateTablet for tablet ec789761c5344074974b9c23b79e418a (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:46.848938 23265 catalog_manager.cc:5697] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:46.826294 23639 tablet_bootstrap.cc:654] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.852926 23370 raft_consensus.cc:2468] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:46.852921 23677 consensus_queue.cc:1048] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:46.854117 23355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e8e3d7d4343c88e56c92382808e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.853781 23643 raft_consensus.cc:515] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:46.865331 23428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d36e08707f4d8295154d67b1450b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.876192 23691 raft_consensus.cc:493] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.877040 23484 leader_election.cc:304] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:46.878098 23645 ts_tablet_manager.cc:1403] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.291s	user 0.004s	sys 0.023s
I20260709 13:06:46.891474 23638 ts_tablet_manager.cc:1434] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.220s	user 0.010s	sys 0.002s
W20260709 13:06:46.898108 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 46595.6us
I20260709 13:06:46.884322 23690 consensus_queue.cc:1048] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:06:46.891214 23691 raft_consensus.cc:515] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.906095 23638 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:46.907501 23691 leader_election.cc:290] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:46.892372 23669 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:06:46.899111 23359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec789761c5344074974b9c23b79e418a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:46.898696 23333 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:46.929359 23644 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:46.929461 23356 tablet_service.cc:1511] Processing CreateTablet for tablet ae7dba7096144600b277ae19d383145e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:46.931221 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:46.932006 23370 raft_consensus.cc:2393] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceb56da8fd5145648a898f5487219c6e in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:46.932967 23354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3067bf3f594c4985b356c81a410f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.933656 23356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7dba7096144600b277ae19d383145e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:46.922853 23484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 45892.8us
I20260709 13:06:46.934401 23353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf62d6f4e045f8b06e7170496fda73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.944370 23661 raft_consensus.cc:2804] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:46.944891 23661 raft_consensus.cc:493] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.945226 23661 raft_consensus.cc:3060] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.950246 23582 tablet_service.cc:1511] Processing CreateTablet for tablet 37d36e08707f4d8295154d67b1450b0f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 13:06:46.923767 23482 leader_election.cc:343] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:46.958597 23643 leader_election.cc:290] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:46.953423 23645 raft_consensus.cc:359] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:46.971138 23645 raft_consensus.cc:385] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.971504 23645 raft_consensus.cc:740] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.987914 23434 tablet_service.cc:1511] Processing CreateTablet for tablet c366e593434e49a9b0f8d3a41616077d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:46.989559 23434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c366e593434e49a9b0f8d3a41616077d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.993597 23582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d36e08707f4d8295154d67b1450b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.994607 23575 tablet_service.cc:1511] Processing CreateTablet for tablet c366e593434e49a9b0f8d3a41616077d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:46.996025 23575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c366e593434e49a9b0f8d3a41616077d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:46.966759 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 44000.3us
I20260709 13:06:46.996588 23580 tablet_service.cc:1511] Processing CreateTablet for tablet 0675ebf2d38f494d9b749b40e54bea4c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:46.998483 23580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675ebf2d38f494d9b749b40e54bea4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.000613 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:47.000813 23507 tablet_service.cc:1511] Processing CreateTablet for tablet 0675ebf2d38f494d9b749b40e54bea4c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:47.002344 23507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675ebf2d38f494d9b749b40e54bea4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.003121 23505 tablet_service.cc:1511] Processing CreateTablet for tablet 37d36e08707f4d8295154d67b1450b0f (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:47.004498 23505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d36e08707f4d8295154d67b1450b0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.013090 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.014922 23642 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.015779 23642 ts_tablet_manager.cc:1403] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.503s	user 0.017s	sys 0.008s
I20260709 13:06:47.019348 23357 tablet_service.cc:1511] Processing CreateTablet for tablet c366e593434e49a9b0f8d3a41616077d (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:47.022447 23707 consensus_queue.cc:1048] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:47.006165 23502 tablet_service.cc:1511] Processing CreateTablet for tablet ae7dba7096144600b277ae19d383145e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:47.026005 23502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7dba7096144600b277ae19d383145e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.026908 23645 consensus_queue.cc:260] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.036830 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c366e593434e49a9b0f8d3a41616077d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.042533 23645 ts_tablet_manager.cc:1434] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.155s	user 0.005s	sys 0.002s
I20260709 13:06:47.043593 23645 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
W20260709 13:06:47.031335 23482 leader_election.cc:343] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.051074 23435 tablet_service.cc:1511] Processing CreateTablet for tablet ae7dba7096144600b277ae19d383145e (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:47.052654 23360 tablet_service.cc:1511] Processing CreateTablet for tablet 0675ebf2d38f494d9b749b40e54bea4c (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:47.059984 23360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675ebf2d38f494d9b749b40e54bea4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.069837 23642 raft_consensus.cc:359] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.073369 23642 raft_consensus.cc:385] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.073756 23642 raft_consensus.cc:740] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.074697 23642 consensus_queue.cc:260] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.077658 23642 ts_tablet_manager.cc:1434] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 13:06:47.081770 23642 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.090106 23593 raft_consensus.cc:3060] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:47.090510 23482 leader_election.cc:343] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.090997 23482 leader_election.cc:304] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:47.092106 23713 raft_consensus.cc:2749] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:47.052700 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7dba7096144600b277ae19d383145e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.099035 23713 consensus_queue.cc:1048] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:47.108717 23639 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.109365 23639 ts_tablet_manager.cc:1403] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.346s	user 0.007s	sys 0.020s
I20260709 13:06:47.133872 23432 tablet_service.cc:1511] Processing CreateTablet for tablet 5e66e5d5789d4dc3a56f76f34b603bad (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:47.135257 23432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e66e5d5789d4dc3a56f76f34b603bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.137076 23691 raft_consensus.cc:493] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.137660 23691 raft_consensus.cc:515] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.139778 23691 leader_election.cc:290] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.141157 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.142007 23370 raft_consensus.cc:2468] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:47.143402 23558 leader_election.cc:304] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.150997 23645 tablet_bootstrap.cc:654] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.155462 23503 tablet_service.cc:1511] Processing CreateTablet for tablet 5e66e5d5789d4dc3a56f76f34b603bad (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:47.157406 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e66e5d5789d4dc3a56f76f34b603bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.158047 23691 raft_consensus.cc:2804] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.158538 23691 raft_consensus.cc:493] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.158950 23691 raft_consensus.cc:3060] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.159641 23638 tablet_bootstrap.cc:654] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.161705 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.162531 23590 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:47.164244 23333 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:47.165099 23644 raft_consensus.cc:2804] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.165480 23644 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.165813 23644 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.166051 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3cf23d64f04f4cb98bbd43988743ea" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.167876 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.169837 23556 leader_election.cc:304] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:47.084381 23661 raft_consensus.cc:515] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.190218 23519 raft_consensus.cc:2393] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceb56da8fd5145648a898f5487219c6e in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:47.192744 23576 tablet_service.cc:1511] Processing CreateTablet for tablet 5e66e5d5789d4dc3a56f76f34b603bad (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:47.194591 23576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e66e5d5789d4dc3a56f76f34b603bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.199800 23431 tablet_service.cc:1511] Processing CreateTablet for tablet 64b73b48f1244e19a12b43c7f7a82cc5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:47.201248 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b73b48f1244e19a12b43c7f7a82cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.112747 23639 raft_consensus.cc:359] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.211490 23639 raft_consensus.cc:385] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.217238 23639 raft_consensus.cc:740] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.218044 23639 consensus_queue.cc:260] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.220979 23639 ts_tablet_manager.cc:1434] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.002s
I20260709 13:06:47.220624 23502 tablet_service.cc:1511] Processing CreateTablet for tablet 64b73b48f1244e19a12b43c7f7a82cc5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:47.222092 23502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b73b48f1244e19a12b43c7f7a82cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.222132 23639 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.230489 23357 tablet_service.cc:1511] Processing CreateTablet for tablet 64b73b48f1244e19a12b43c7f7a82cc5 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:47.235525 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b73b48f1244e19a12b43c7f7a82cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.241850 23661 leader_election.cc:290] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.246151 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:47.246927 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:47.250808 23370 raft_consensus.cc:3060] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:47.251965 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:47.249190 23577 tablet_service.cc:1511] Processing CreateTablet for tablet ce891b8160664205892014cec10858ff (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:47.257647 23577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce891b8160664205892014cec10858ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:47.234468 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 64769.5us
I20260709 13:06:47.267810 23644 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.272107 23642 tablet_bootstrap.cc:654] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.275645 23639 tablet_bootstrap.cc:654] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.289855 23431 tablet_service.cc:1511] Processing CreateTablet for tablet ce891b8160664205892014cec10858ff (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:47.291371 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce891b8160664205892014cec10858ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.295013 23575 tablet_service.cc:1511] Processing CreateTablet for tablet 787a4af7968d4c359c9efbc9abfe9b40 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:47.296280 23575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787a4af7968d4c359c9efbc9abfe9b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.290810 23691 raft_consensus.cc:515] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.293087 23505 tablet_service.cc:1511] Processing CreateTablet for tablet 787a4af7968d4c359c9efbc9abfe9b40 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:47.298146 23505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787a4af7968d4c359c9efbc9abfe9b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.289855 23354 tablet_service.cc:1511] Processing CreateTablet for tablet ce891b8160664205892014cec10858ff (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:47.305080 23638 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.305668 23638 ts_tablet_manager.cc:1403] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.400s	user 0.009s	sys 0.012s
I20260709 13:06:47.305716 23357 tablet_service.cc:1511] Processing CreateTablet for tablet 787a4af7968d4c359c9efbc9abfe9b40 (DEFAULT_TABLE table=test-workload [id=13f244d2bc9e43039318c868d26ef8e0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:47.306973 23357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787a4af7968d4c359c9efbc9abfe9b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.308192 23638 raft_consensus.cc:359] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.298023 23370 raft_consensus.cc:2468] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:47.310032 23638 raft_consensus.cc:385] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.336999 23638 raft_consensus.cc:740] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.337847 23638 consensus_queue.cc:260] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.340181 23638 ts_tablet_manager.cc:1434] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:06:47.331761 23724 raft_consensus.cc:2749] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:47.333766 23642 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.334194 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:47.343204 23484 leader_election.cc:304] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.343329 23591 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.345760 23713 raft_consensus.cc:2804] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.346253 23713 raft_consensus.cc:697] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:47.347074 23713 consensus_queue.cc:237] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.349102 23354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce891b8160664205892014cec10858ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.356681 23591 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:47.357929 23691 leader_election.cc:290] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.358731 23333 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.359529 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:47.360316 23370 raft_consensus.cc:2393] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ceb56da8fd5145648a898f5487219c6e in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:47.361335 23728 raft_consensus.cc:2804] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.364701 23642 ts_tablet_manager.cc:1403] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.283s	user 0.010s	sys 0.006s
I20260709 13:06:47.369874 23592 raft_consensus.cc:2393] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:47.363878 23644 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:47.381278 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:47.378295 23645 tablet_bootstrap.cc:492] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
W20260709 13:06:47.396957 23333 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.398943 23638 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.387977 23642 raft_consensus.cc:359] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.376259 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c16a9c7d17249d98740f56a32c03d84" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:47.407984 23645 ts_tablet_manager.cc:1403] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.365s	user 0.021s	sys 0.011s
I20260709 13:06:47.407194 23642 raft_consensus.cc:385] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.416435 23556 leader_election.cc:304] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:47.417685 23724 raft_consensus.cc:2749] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:47.417886 23645 raft_consensus.cc:359] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.418638 23645 raft_consensus.cc:385] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.418936 23645 raft_consensus.cc:740] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.419696 23645 consensus_queue.cc:260] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.412492 23642 raft_consensus.cc:740] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.422015 23642 consensus_queue.cc:260] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.424156 23645 ts_tablet_manager.cc:1434] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:47.426819 23639 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.427326 23639 ts_tablet_manager.cc:1403] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.205s	user 0.016s	sys 0.001s
I20260709 13:06:47.429900 23639 raft_consensus.cc:359] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.430421 23639 raft_consensus.cc:385] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.430668 23639 raft_consensus.cc:740] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.431217 23639 consensus_queue.cc:260] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.433123 23638 tablet_bootstrap.cc:654] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.436864 23639 ts_tablet_manager.cc:1434] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:06:47.437872 23639 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.441475 23642 ts_tablet_manager.cc:1434] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
I20260709 13:06:47.443508 23639 tablet_bootstrap.cc:654] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.411139 23728 raft_consensus.cc:697] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:47.453878 23728 consensus_queue.cc:237] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.450429 23642 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.444041 23267 catalog_manager.cc:5697] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:47.465525 23645 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.472908 23690 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.476259 23268 catalog_manager.cc:5697] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:47.477389 23690 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.480181 23639 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.480724 23639 ts_tablet_manager.cc:1403] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.015s
I20260709 13:06:47.480594 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.486851 23638 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.487340 23638 ts_tablet_manager.cc:1403] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.089s	user 0.004s	sys 0.028s
I20260709 13:06:47.486644 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.488303 23592 raft_consensus.cc:2393] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:47.489192 23642 tablet_bootstrap.cc:654] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.489617 23408 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:47.485242 23690 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.493405 23677 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.490903 23638 raft_consensus.cc:359] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.494506 23638 raft_consensus.cc:385] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.494870 23638 raft_consensus.cc:740] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.495333 23645 tablet_bootstrap.cc:654] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.491925 23639 raft_consensus.cc:359] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.495883 23638 consensus_queue.cc:260] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.498252 23638 ts_tablet_manager.cc:1434] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:06:47.495927 23639 raft_consensus.cc:385] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.502075 23677 raft_consensus.cc:2749] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:47.498963 23728 raft_consensus.cc:493] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.502911 23639 raft_consensus.cc:740] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.503126 23728 raft_consensus.cc:515] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.504300 23639 consensus_queue.cc:260] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.506240 23710 raft_consensus.cc:493] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.506945 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:47.508394 23638 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.509020 23710 raft_consensus.cc:515] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.511480 23710 leader_election.cc:290] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:47.513247 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.513993 23519 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
W20260709 13:06:47.514739 23333 leader_election.cc:343] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.515442 23728 leader_election.cc:290] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:47.515509 23408 leader_election.cc:304] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:47.513818 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.516991 23638 tablet_bootstrap.cc:654] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.517335 23592 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:47.508298 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.518792 23518 raft_consensus.cc:2468] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:47.520112 23333 leader_election.cc:304] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:47.520963 23639 ts_tablet_manager.cc:1434] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 13:06:47.521240 23728 raft_consensus.cc:2804] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.521600 23728 raft_consensus.cc:493] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.521907 23728 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.521955 23639 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.520977 23642 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.522783 23642 ts_tablet_manager.cc:1403] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.073s	user 0.016s	sys 0.007s
I20260709 13:06:47.525796 23710 raft_consensus.cc:2804] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.529140 23710 raft_consensus.cc:493] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.525853 23642 raft_consensus.cc:359] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.528705 23639 tablet_bootstrap.cc:654] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.529533 23710 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.529894 23642 raft_consensus.cc:385] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.530318 23642 raft_consensus.cc:740] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.531315 23642 consensus_queue.cc:260] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.531512 23728 raft_consensus.cc:515] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.534658 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:47.535246 23445 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.536698 23642 ts_tablet_manager.cc:1434] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:47.538039 23642 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.540073 23645 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:47.540599 23645 ts_tablet_manager.cc:1403] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.012s
I20260709 13:06:47.540398 23710 raft_consensus.cc:515] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.542567 23710 leader_election.cc:290] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:47.543955 23645 raft_consensus.cc:359] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.544713 23645 raft_consensus.cc:385] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.544661 23642 tablet_bootstrap.cc:654] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.547433 23638 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.547943 23638 ts_tablet_manager.cc:1403] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.006s
I20260709 13:06:47.551604 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:47.551570 23638 raft_consensus.cc:359] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.545645 23645 raft_consensus.cc:740] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.555039 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:47.554308 23645 consensus_queue.cc:260] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.556067 23592 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.558053 23645 ts_tablet_manager.cc:1434] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:06:47.559171 23645 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.552663 23638 raft_consensus.cc:385] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.561249 23642 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.561498 23638 raft_consensus.cc:740] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.561746 23642 ts_tablet_manager.cc:1403] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.000s
I20260709 13:06:47.562507 23638 consensus_queue.cc:260] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.564646 23642 raft_consensus.cc:359] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.565291 23638 ts_tablet_manager.cc:1434] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:06:47.565572 23642 raft_consensus.cc:385] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.565968 23642 raft_consensus.cc:740] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.566303 23638 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.566738 23642 consensus_queue.cc:260] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.568758 23642 ts_tablet_manager.cc:1434] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:47.569731 23642 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.572880 23638 tablet_bootstrap.cc:654] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.575871 23642 tablet_bootstrap.cc:654] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.552699 23518 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.587419 23645 tablet_bootstrap.cc:654] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.595712 23518 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:47.598165 23408 leader_election.cc:304] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:47.599320 23710 raft_consensus.cc:2804] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.599947 23710 raft_consensus.cc:697] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:47.600320 23724 raft_consensus.cc:493] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.601040 23724 raft_consensus.cc:515] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.601604 23710 consensus_queue.cc:237] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.604121 23724 leader_election.cc:290] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.605811 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "893c1c5680b947488f9d810e532fe7e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.606683 23370 raft_consensus.cc:2468] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:47.607784 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "893c1c5680b947488f9d810e532fe7e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.608155 23558 leader_election.cc:304] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.609505 23724 raft_consensus.cc:2804] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.610025 23724 raft_consensus.cc:493] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.610538 23724 raft_consensus.cc:3060] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.623106 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:47.624136 23519 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.623202 23638 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.627918 23638 ts_tablet_manager.cc:1403] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.001s
I20260709 13:06:47.631732 23518 raft_consensus.cc:2468] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:47.639619 23645 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:47.640306 23645 ts_tablet_manager.cc:1403] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.005s
I20260709 13:06:47.636628 23445 raft_consensus.cc:2468] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:47.644701 23638 raft_consensus.cc:359] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.645772 23638 raft_consensus.cc:385] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.646216 23638 raft_consensus.cc:740] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.648111 23645 raft_consensus.cc:359] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.649150 23645 raft_consensus.cc:385] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.649506 23645 raft_consensus.cc:740] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.650403 23724 raft_consensus.cc:515] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.654690 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "893c1c5680b947488f9d810e532fe7e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:47.655597 23518 raft_consensus.cc:3060] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.661231 23638 consensus_queue.cc:260] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.664817 23724 leader_election.cc:290] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.665802 23645 consensus_queue.cc:260] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.667411 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "893c1c5680b947488f9d810e532fe7e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:47.668025 23370 raft_consensus.cc:3060] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.671968 23518 raft_consensus.cc:2468] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:47.673333 23556 leader_election.cc:304] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:47.674496 23691 raft_consensus.cc:2804] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.675051 23691 raft_consensus.cc:697] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:47.675503 23370 raft_consensus.cc:2468] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:47.675966 23691 consensus_queue.cc:237] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.682320 23638 ts_tablet_manager.cc:1434] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.054s	user 0.010s	sys 0.000s
I20260709 13:06:47.683624 23638 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.674103 23267 catalog_manager.cc:5697] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:47.685516 23645 ts_tablet_manager.cc:1434] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 13:06:47.640144 23728 leader_election.cc:290] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:47.687028 23645 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.687845 23333 leader_election.cc:304] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.689508 23728 raft_consensus.cc:2804] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.690634 23728 raft_consensus.cc:697] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:47.692061 23728 consensus_queue.cc:237] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.699261 23519 raft_consensus.cc:2468] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:47.700484 23638 tablet_bootstrap.cc:654] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.704034 23639 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.702114 23592 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:47.708431 23639 ts_tablet_manager.cc:1403] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.187s	user 0.071s	sys 0.095s
I20260709 13:06:47.708945 23642 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.709450 23642 ts_tablet_manager.cc:1403] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.140s	user 0.069s	sys 0.048s
I20260709 13:06:47.707821 23645 tablet_bootstrap.cc:654] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.711364 23639 raft_consensus.cc:359] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.712090 23639 raft_consensus.cc:385] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.712404 23639 raft_consensus.cc:740] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.712173 23642 raft_consensus.cc:359] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.712921 23642 raft_consensus.cc:385] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.713196 23642 raft_consensus.cc:740] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.713174 23639 consensus_queue.cc:260] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.715431 23639 ts_tablet_manager.cc:1434] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:06:47.723516 23639 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.719434 23642 consensus_queue.cc:260] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.725490 23642 ts_tablet_manager.cc:1434] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:06:47.727626 23642 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.730751 23268 catalog_manager.cc:5697] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:47.733549 23642 tablet_bootstrap.cc:654] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.737429 23639 tablet_bootstrap.cc:654] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.743208 23638 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.743743 23638 ts_tablet_manager.cc:1403] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.000s
I20260709 13:06:47.749238 23638 raft_consensus.cc:359] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.750180 23638 raft_consensus.cc:385] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.750530 23638 raft_consensus.cc:740] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.753561 23266 catalog_manager.cc:5697] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:47.758576 23638 consensus_queue.cc:260] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.761152 23638 ts_tablet_manager.cc:1434] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:06:47.761735 23642 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:47.762295 23642 ts_tablet_manager.cc:1403] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.014s
I20260709 13:06:47.762310 23638 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.763211 23639 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.763681 23639 ts_tablet_manager.cc:1403] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.008s
I20260709 13:06:47.767220 23713 consensus_queue.cc:1048] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:47.767328 23645 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:47.768348 23645 ts_tablet_manager.cc:1403] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.009s
I20260709 13:06:47.766508 23639 raft_consensus.cc:359] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.771255 23645 raft_consensus.cc:359] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.765245 23642 raft_consensus.cc:359] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.772385 23645 raft_consensus.cc:385] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.772740 23645 raft_consensus.cc:740] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.772677 23642 raft_consensus.cc:385] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.773335 23642 raft_consensus.cc:740] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.771297 23639 raft_consensus.cc:385] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.779318 23639 raft_consensus.cc:740] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.775110 23645 consensus_queue.cc:260] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.780025 23639 consensus_queue.cc:260] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.782035 23639 ts_tablet_manager.cc:1434] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:06:47.778789 23638 tablet_bootstrap.cc:654] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.784595 23645 ts_tablet_manager.cc:1434] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:06:47.785697 23639 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.786791 23707 consensus_queue.cc:1048] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:47.774531 23642 consensus_queue.cc:260] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.798584 23642 ts_tablet_manager.cc:1434] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:06:47.800094 23642 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:47.795867 23639 tablet_bootstrap.cc:654] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.785691 23645 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.807588 23642 tablet_bootstrap.cc:654] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.811847 23645 tablet_bootstrap.cc:654] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.821060 23645 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:47.821062 23638 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.821730 23645 ts_tablet_manager.cc:1403] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.005s
I20260709 13:06:47.821885 23638 ts_tablet_manager.cc:1403] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.014s
I20260709 13:06:47.824579 23639 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.824628 23645 raft_consensus.cc:359] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.825315 23639 ts_tablet_manager.cc:1403] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.000s
I20260709 13:06:47.825613 23645 raft_consensus.cc:385] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.825783 23728 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.825985 23645 raft_consensus.cc:740] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.826056 23638 raft_consensus.cc:359] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.826315 23728 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.826843 23645 consensus_queue.cc:260] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.826861 23638 raft_consensus.cc:385] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.828095 23638 raft_consensus.cc:740] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.828792 23638 consensus_queue.cc:260] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.829257 23639 raft_consensus.cc:359] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.830119 23639 raft_consensus.cc:385] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.830307 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.830493 23639 raft_consensus.cc:740] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.830979 23638 ts_tablet_manager.cc:1434] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:47.831324 23736 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.832012 23638 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.831569 23639 consensus_queue.cc:260] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.831833 23736 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:47.833817 23333 leader_election.cc:343] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.833943 23639 ts_tablet_manager.cc:1434] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:47.834075 23736 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:47.830538 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:47.835172 23639 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.835152 23445 raft_consensus.cc:2468] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:47.837975 23645 ts_tablet_manager.cc:1434] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:06:47.838860 23638 tablet_bootstrap.cc:654] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.839180 23645 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.828557 23728 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:47.841457 23639 tablet_bootstrap.cc:654] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.847476 23645 tablet_bootstrap.cc:654] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.853281 23333 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:47.853425 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:47.854628 23638 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.853494 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:47.855163 23638 ts_tablet_manager.cc:1403] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:06:47.855669 23519 raft_consensus.cc:2468] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:47.858012 23736 raft_consensus.cc:2804] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.858500 23736 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.858872 23736 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:47.858979 23333 leader_election.cc:343] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.860143 23333 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:47.862440 23644 raft_consensus.cc:2804] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.862905 23644 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.863262 23644 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.869223 23638 raft_consensus.cc:359] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.870086 23638 raft_consensus.cc:385] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.870411 23638 raft_consensus.cc:740] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.871255 23638 consensus_queue.cc:260] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.873283 23736 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.875979 23736 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:47.878509 23713 raft_consensus.cc:493] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.879676 23638 ts_tablet_manager.cc:1434] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:06:47.881223 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:47.881549 23638 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.881624 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:47.882747 23445 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.879447 23713 raft_consensus.cc:515] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.888897 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
W20260709 13:06:47.890919 23482 leader_election.cc:343] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:47.893680 23333 leader_election.cc:343] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.896935 23639 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:47.897497 23639 ts_tablet_manager.cc:1403] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.000s
I20260709 13:06:47.897661 23638 tablet_bootstrap.cc:654] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.897792 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.898758 23591 raft_consensus.cc:2468] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:47.900278 23445 raft_consensus.cc:2468] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:47.900297 23482 leader_election.cc:304] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:47.900875 23639 raft_consensus.cc:359] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.901618 23639 raft_consensus.cc:385] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.901777 23707 raft_consensus.cc:2804] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.902616 23707 raft_consensus.cc:493] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.903110 23707 raft_consensus.cc:3060] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.904412 23645 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:47.904955 23645 ts_tablet_manager.cc:1403] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.003s
I20260709 13:06:47.901894 23639 raft_consensus.cc:740] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.901873 23333 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:47.910290 23736 raft_consensus.cc:2804] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.911156 23661 raft_consensus.cc:493] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.911727 23661 raft_consensus.cc:515] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.908799 23645 raft_consensus.cc:359] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.912705 23736 raft_consensus.cc:697] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:47.914101 23736 consensus_queue.cc:237] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.909730 23638 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:47.916213 23638 ts_tablet_manager.cc:1403] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.008s
I20260709 13:06:47.916332 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.917186 23370 raft_consensus.cc:2468] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:47.918584 23645 raft_consensus.cc:385] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.919023 23645 raft_consensus.cc:740] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.919364 23638 raft_consensus.cc:359] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.920140 23638 raft_consensus.cc:385] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.920312 23484 leader_election.cc:304] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.919906 23645 consensus_queue.cc:260] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.921567 23661 leader_election.cc:290] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.922200 23661 raft_consensus.cc:2804] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.922595 23661 raft_consensus.cc:493] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.923050 23661 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.928212 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.910161 23639 consensus_queue.cc:260] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
W20260709 13:06:47.929999 23482 leader_election.cc:343] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.928181 23645 ts_tablet_manager.cc:1434] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:06:47.931308 23639 ts_tablet_manager.cc:1434] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:06:47.932344 23639 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:47.933326 23736 consensus_queue.cc:1048] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:47.936367 23710 raft_consensus.cc:493] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.937005 23710 raft_consensus.cc:515] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.939342 23710 leader_election.cc:290] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.940150 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:47.941001 23593 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:47.943507 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.932243 23645 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:47.943984 23772 consensus_queue.cc:1048] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:47.944227 23369 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:47.946130 23410 leader_election.cc:304] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:47.920514 23638 raft_consensus.cc:740] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.948487 23677 raft_consensus.cc:2804] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:47.950274 23677 raft_consensus.cc:493] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.950695 23677 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.950475 23638 consensus_queue.cc:260] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.953285 23638 ts_tablet_manager.cc:1434] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:06:47.954319 23638 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:47.956966 23645 tablet_bootstrap.cc:654] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.972198 23691 raft_consensus.cc:493] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.944749 23661 raft_consensus.cc:515] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:47.972817 23691 raft_consensus.cc:515] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.975752 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:47.976342 23691 leader_election.cc:290] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:47.976492 23661 leader_election.cc:290] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.977120 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:47.978485 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
W20260709 13:06:47.978648 23558 leader_election.cc:343] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.982092 23639 tablet_bootstrap.cc:654] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:47.984601 23482 leader_election.cc:343] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.960552 23713 leader_election.cc:290] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
W20260709 13:06:47.986876 23556 leader_election.cc:343] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.988870 23638 tablet_bootstrap.cc:654] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.989306 23556 leader_election.cc:304] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:47.989373 23644 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:47.991125 23728 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.991704 23691 raft_consensus.cc:2749] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:47.973985 23707 raft_consensus.cc:515] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:47.996363 23677 raft_consensus.cc:515] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:47.996973 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
W20260709 13:06:47.998513 23333 leader_election.cc:343] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:47.998766 23677 leader_election.cc:290] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:47.997917 23644 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:47.991712 23728 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.000155 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.001103 23592 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.002104 23728 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.010927 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:48.011873 23370 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.012529 23592 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:48.014393 23408 leader_election.cc:304] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:48.015609 23677 raft_consensus.cc:2804] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.016216 23677 raft_consensus.cc:697] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:48.017297 23677 consensus_queue.cc:237] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.021373 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.022351 23444 raft_consensus.cc:2468] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:48.025978 23690 consensus_queue.cc:1048] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.042177 23333 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:48.043973 23644 raft_consensus.cc:2804] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.047605 23644 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.047979 23644 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:48.039057 23484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 118837us
I20260709 13:06:48.050453 23691 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.051016 23691 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.055521 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
W20260709 13:06:48.057101 23482 leader_election.cc:343] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.059262 23645 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.059828 23645 ts_tablet_manager.cc:1403] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.128s	user 0.015s	sys 0.007s
I20260709 13:06:48.060552 23642 tablet_bootstrap.cc:492] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.061111 23642 ts_tablet_manager.cc:1403] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.261s	user 0.052s	sys 0.118s
I20260709 13:06:48.062799 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:48.063226 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.062212 23691 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.064021 23444 raft_consensus.cc:2393] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceb56da8fd5145648a898f5487219c6e in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:48.066710 23370 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:48.068954 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:48.069175 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:48.069591 23369 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.069811 23519 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.065227 23556 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:48.052943 23771 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.077634 23771 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.079042 23519 raft_consensus.cc:2468] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:48.080430 23771 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.081434 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.081967 23707 leader_election.cc:290] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.082116 23444 raft_consensus.cc:2468] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:48.082201 23724 consensus_queue.cc:1048] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.083603 23482 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:48.084442 23639 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.073567 23691 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.085042 23707 raft_consensus.cc:2804] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.085446 23707 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.085999 23707 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.083619 23642 raft_consensus.cc:359] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.088511 23639 ts_tablet_manager.cc:1403] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.156s	user 0.013s	sys 0.002s
I20260709 13:06:48.088703 23642 raft_consensus.cc:385] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.089083 23642 raft_consensus.cc:740] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.082839 23638 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.092371 23638 ts_tablet_manager.cc:1403] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.138s	user 0.010s	sys 0.018s
I20260709 13:06:48.091686 23266 catalog_manager.cc:5697] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.093744 23639 raft_consensus.cc:359] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.095745 23639 raft_consensus.cc:385] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.096464 23639 raft_consensus.cc:740] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.096626 23645 raft_consensus.cc:359] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.097404 23645 raft_consensus.cc:385] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.097694 23645 raft_consensus.cc:740] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.098165 23639 consensus_queue.cc:260] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.101293 23639 ts_tablet_manager.cc:1434] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:06:48.107721 23639 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:48.109603 23691 raft_consensus.cc:2749] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.098364 23645 consensus_queue.cc:260] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.103134 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:48.091504 23642 consensus_queue.cc:260] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.104169 23333 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:48.112036 23642 ts_tablet_manager.cc:1434] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:06:48.112514 23736 raft_consensus.cc:2804] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.113057 23736 raft_consensus.cc:697] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:48.113278 23642 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.113938 23736 consensus_queue.cc:237] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.101296 23638 raft_consensus.cc:359] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.117444 23638 raft_consensus.cc:385] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.117761 23638 raft_consensus.cc:740] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.118462 23638 consensus_queue.cc:260] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.120102 23642 tablet_bootstrap.cc:654] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.120709 23638 ts_tablet_manager.cc:1434] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.000s
I20260709 13:06:48.121973 23638 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.128337 23638 tablet_bootstrap.cc:654] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.128544 23267 catalog_manager.cc:5697] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.130612 23707 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.112036 23645 ts_tablet_manager.cc:1434] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.000s
I20260709 13:06:48.133708 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:48.082734 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.135991 23519 raft_consensus.cc:2393] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:48.136080 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.136708 23591 raft_consensus.cc:3060] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.143407 23642 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.140604 23645 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.148774 23642 ts_tablet_manager.cc:1403] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.005s
I20260709 13:06:48.147135 23639 tablet_bootstrap.cc:654] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.157021 23707 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.158432 23644 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.160184 23367 raft_consensus.cc:2393] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:48.155680 23369 raft_consensus.cc:2468] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.165328 23484 leader_election.cc:304] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.166109 23707 raft_consensus.cc:2804] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.163578 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:48.166618 23707 raft_consensus.cc:697] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:48.166862 23445 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.167536 23707 consensus_queue.cc:237] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.172930 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:48.174525 23482 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:48.175684 23771 raft_consensus.cc:2749] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.179091 23644 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.180231 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:48.181115 23519 raft_consensus.cc:2393] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:48.183734 23726 consensus_queue.cc:1048] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.186236 23690 consensus_queue.cc:1048] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.191190 23642 raft_consensus.cc:359] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.192103 23642 raft_consensus.cc:385] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.192441 23642 raft_consensus.cc:740] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.168344 23591 raft_consensus.cc:2468] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.199064 23482 leader_election.cc:304] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:48.201356 23771 raft_consensus.cc:2804] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.201905 23771 raft_consensus.cc:697] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:48.202298 23445 raft_consensus.cc:2468] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:48.200827 23642 consensus_queue.cc:260] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.202795 23771 consensus_queue.cc:237] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.204099 23333 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:48.204929 23644 raft_consensus.cc:2804] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.205390 23644 raft_consensus.cc:697] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:48.207743 23645 tablet_bootstrap.cc:654] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.211097 23642 ts_tablet_manager.cc:1434] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:06:48.211648 23644 consensus_queue.cc:237] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.213449 23642 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.214375 23661 raft_consensus.cc:493] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.214892 23661 raft_consensus.cc:515] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.216962 23661 leader_election.cc:290] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.220058 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:48.221678 23482 leader_election.cc:343] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.244997 23642 tablet_bootstrap.cc:654] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.244187 23267 catalog_manager.cc:5697] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.250087 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.252431 23690 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.252972 23690 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:48.254190 23482 leader_election.cc:343] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.254657 23482 leader_election.cc:304] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.255048 23690 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.256580 23639 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.256704 23661 raft_consensus.cc:2749] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.256906 23638 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.257046 23639 ts_tablet_manager.cc:1403] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.001s
I20260709 13:06:48.257377 23638 ts_tablet_manager.cc:1403] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.136s	user 0.017s	sys 0.000s
I20260709 13:06:48.254967 23645 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.260283 23645 ts_tablet_manager.cc:1403] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.003s
I20260709 13:06:48.262256 23639 raft_consensus.cc:359] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.261301 23638 raft_consensus.cc:359] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.263916 23638 raft_consensus.cc:385] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.264333 23638 raft_consensus.cc:740] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.265532 23638 consensus_queue.cc:260] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.262282 23771 raft_consensus.cc:493] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.263177 23639 raft_consensus.cc:385] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.268285 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.273083 23639 raft_consensus.cc:740] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.273495 23518 raft_consensus.cc:2393] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:48.274097 23639 consensus_queue.cc:260] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.268502 23638 ts_tablet_manager.cc:1434] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:48.265885 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:48.260597 23644 consensus_queue.cc:1048] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:06:48.276290 23638 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.280401 23410 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:48.281875 23690 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.285318 23736 consensus_queue.cc:1048] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.287180 23639 ts_tablet_manager.cc:1434] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.005s
I20260709 13:06:48.288028 23639 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
W20260709 13:06:48.289201 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:48.297116 23639 tablet_bootstrap.cc:654] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.298969 23265 catalog_manager.cc:5697] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.313309 23645 raft_consensus.cc:359] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.314450 23645 raft_consensus.cc:385] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.314822 23645 raft_consensus.cc:740] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.321481 23266 catalog_manager.cc:5697] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.325817 23642 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.287062 23771 raft_consensus.cc:515] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.326399 23642 ts_tablet_manager.cc:1403] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.114s	user 0.026s	sys 0.012s
I20260709 13:06:48.328084 23736 raft_consensus.cc:493] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.328611 23736 raft_consensus.cc:515] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.329413 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4a932d15eb4e9e89e5b9bf6627de2b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.300524 23690 raft_consensus.cc:2749] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.330103 23591 raft_consensus.cc:2468] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:48.331630 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4a932d15eb4e9e89e5b9bf6627de2b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.331928 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.332710 23482 leader_election.cc:304] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:48.332821 23519 raft_consensus.cc:2468] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:48.333654 23713 raft_consensus.cc:2804] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.334070 23713 raft_consensus.cc:493] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.334424 23713 raft_consensus.cc:3060] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.334601 23333 leader_election.cc:304] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:48.328224 23771 leader_election.cc:290] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.337085 23644 raft_consensus.cc:2804] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.337597 23644 raft_consensus.cc:493] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.338050 23644 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.329257 23642 raft_consensus.cc:359] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.341590 23642 raft_consensus.cc:385] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.341892 23642 raft_consensus.cc:740] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.342689 23642 consensus_queue.cc:260] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.344779 23642 ts_tablet_manager.cc:1434] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
W20260709 13:06:48.346279 23482 leader_election.cc:343] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.347649 23638 tablet_bootstrap.cc:654] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.348165 23736 leader_election.cc:290] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.348691 23642 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.349318 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:48.350651 23333 leader_election.cc:343] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.354084 23644 raft_consensus.cc:515] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.356237 23644 leader_election.cc:290] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.347254 23713 raft_consensus.cc:515] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.357474 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.357527 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:48.360412 23518 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.361690 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4a932d15eb4e9e89e5b9bf6627de2b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.362358 23591 raft_consensus.cc:3060] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.361690 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc4a932d15eb4e9e89e5b9bf6627de2b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
W20260709 13:06:48.364605 23482 leader_election.cc:343] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:48.366000 23333 leader_election.cc:343] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.368317 23642 tablet_bootstrap.cc:654] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.369885 23645 consensus_queue.cc:260] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.389243 23267 catalog_manager.cc:5697] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.389987 23728 consensus_queue.cc:1048] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.408429 23645 ts_tablet_manager.cc:1434] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.148s	user 0.007s	sys 0.000s
I20260709 13:06:48.358521 23713 leader_election.cc:290] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.428666 23645 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.428498 23728 consensus_queue.cc:1048] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.439172 23638 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.443694 23638 ts_tablet_manager.cc:1403] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.168s	user 0.013s	sys 0.000s
I20260709 13:06:48.424566 23591 raft_consensus.cc:2468] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.452642 23639 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.452478 23638 raft_consensus.cc:359] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.452813 23482 leader_election.cc:304] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:48.453287 23638 raft_consensus.cc:385] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.453243 23639 ts_tablet_manager.cc:1403] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.165s	user 0.035s	sys 0.084s
I20260709 13:06:48.453759 23638 raft_consensus.cc:740] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.454090 23661 raft_consensus.cc:2804] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.454718 23661 raft_consensus.cc:697] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:48.454604 23638 consensus_queue.cc:260] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.424576 23518 raft_consensus.cc:2468] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:48.458308 23661 consensus_queue.cc:237] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.461247 23638 ts_tablet_manager.cc:1434] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:06:48.461756 23690 consensus_queue.cc:1048] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.464063 23642 tablet_bootstrap.cc:492] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.463547 23639 raft_consensus.cc:359] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.480420 23639 raft_consensus.cc:385] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.471076 23333 leader_election.cc:304] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.482180 23644 raft_consensus.cc:2804] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.482798 23644 raft_consensus.cc:697] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:48.483701 23644 consensus_queue.cc:237] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.480779 23639 raft_consensus.cc:740] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.487020 23639 consensus_queue.cc:260] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.489585 23710 consensus_queue.cc:1048] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.491258 23639 ts_tablet_manager.cc:1434] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:06:48.492357 23639 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:48.473134 23645 tablet_bootstrap.cc:654] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.505461 23642 ts_tablet_manager.cc:1403] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.160s	user 0.019s	sys 0.003s
I20260709 13:06:48.507812 23710 raft_consensus.cc:493] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.508395 23710 raft_consensus.cc:515] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.512177 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:48.512184 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.513049 23369 raft_consensus.cc:2468] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:48.513103 23592 raft_consensus.cc:2468] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:48.513621 23639 tablet_bootstrap.cc:654] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.514472 23410 leader_election.cc:304] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:48.515476 23677 raft_consensus.cc:2804] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.515946 23677 raft_consensus.cc:493] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.516340 23677 raft_consensus.cc:3060] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.516991 23710 leader_election.cc:290] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:48.518879 23645 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.519369 23645 ts_tablet_manager.cc:1403] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.091s	user 0.006s	sys 0.005s
I20260709 13:06:48.508517 23642 raft_consensus.cc:359] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.521023 23642 raft_consensus.cc:385] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.520428 23267 catalog_manager.cc:5697] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.521298 23642 raft_consensus.cc:740] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.521874 23644 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.522037 23642 consensus_queue.cc:260] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.522357 23644 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.524251 23642 ts_tablet_manager.cc:1434] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.000s
I20260709 13:06:48.525269 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:48.527076 23333 leader_election.cc:343] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.530366 23638 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.531199 23645 raft_consensus.cc:359] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.531788 23690 raft_consensus.cc:493] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.532390 23690 raft_consensus.cc:515] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.531966 23645 raft_consensus.cc:385] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.535143 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.535562 23645 raft_consensus.cc:740] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.538084 23642 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.539529 23690 leader_election.cc:290] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.540642 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc86fab625684e049569b12b52fdac33" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.541491 23592 raft_consensus.cc:2393] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:48.541339 23645 consensus_queue.cc:260] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.542892 23408 leader_election.cc:304] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.543619 23645 ts_tablet_manager.cc:1434] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:06:48.544034 23690 raft_consensus.cc:3060] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.551087 23639 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.551719 23639 ts_tablet_manager.cc:1403] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 13:06:48.555359 23267 catalog_manager.cc:5697] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.557226 23728 consensus_queue.cc:1048] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:48.544737 23645 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.565362 23639 raft_consensus.cc:359] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.566176 23639 raft_consensus.cc:385] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.566661 23639 raft_consensus.cc:740] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.569268 23642 tablet_bootstrap.cc:654] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.543699 23677 raft_consensus.cc:515] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.570942 23644 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.571434 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.573242 23638 tablet_bootstrap.cc:654] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:48.573589 23333 leader_election.cc:343] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.574666 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.575258 23592 raft_consensus.cc:3060] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.575768 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8118adb799df4bc2adcd22faba9e44d6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:48.576455 23368 raft_consensus.cc:3060] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.576637 23591 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.574083 23333 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.585150 23592 raft_consensus.cc:2468] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:48.586678 23408 leader_election.cc:304] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:48.589046 23677 leader_election.cc:290] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:48.590499 23661 consensus_queue.cc:1048] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.567335 23639 consensus_queue.cc:260] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.593770 23728 raft_consensus.cc:2749] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.590715 23677 raft_consensus.cc:2804] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.596171 23677 raft_consensus.cc:697] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:48.599647 23677 consensus_queue.cc:237] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.603977 23728 consensus_queue.cc:1048] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.601784 23639 ts_tablet_manager.cc:1434] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 13:06:48.605605 23638 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.606120 23638 ts_tablet_manager.cc:1403] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:06:48.608405 23661 consensus_queue.cc:1048] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:06:48.608938 23638 raft_consensus.cc:359] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.623456 23638 raft_consensus.cc:385] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.623736 23638 raft_consensus.cc:740] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.624378 23638 consensus_queue.cc:260] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.626597 23638 ts_tablet_manager.cc:1434] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:06:48.603453 23645 tablet_bootstrap.cc:654] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.627521 23638 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.627612 23736 raft_consensus.cc:493] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.628108 23736 raft_consensus.cc:515] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.631184 23736 leader_election.cc:290] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.633242 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.634230 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.628068 23639 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:48.643049 23690 raft_consensus.cc:2749] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:48.646914 23333 leader_election.cc:343] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:48.647722 23333 leader_election.cc:343] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.648146 23333 leader_election.cc:304] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:48.588467 23368 raft_consensus.cc:2468] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:48.649643 23644 raft_consensus.cc:2749] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.654312 23642 tablet_bootstrap.cc:492] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.654876 23642 ts_tablet_manager.cc:1403] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.117s	user 0.029s	sys 0.039s
I20260709 13:06:48.661623 23645 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.662166 23645 ts_tablet_manager.cc:1403] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.001s
I20260709 13:06:48.667119 23638 tablet_bootstrap.cc:654] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.668090 23645 raft_consensus.cc:359] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.668879 23645 raft_consensus.cc:385] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.669142 23645 raft_consensus.cc:740] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.669203 23642 raft_consensus.cc:359] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.669849 23645 consensus_queue.cc:260] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.672199 23645 ts_tablet_manager.cc:1434] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:48.670228 23642 raft_consensus.cc:385] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.672755 23642 raft_consensus.cc:740] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.673722 23642 consensus_queue.cc:260] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.676141 23642 ts_tablet_manager.cc:1434] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:06:48.677168 23642 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.684983 23645 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.686661 23642 tablet_bootstrap.cc:654] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.686851 23639 tablet_bootstrap.cc:654] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.696256 23645 tablet_bootstrap.cc:654] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.708282 23639 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.708534 23645 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.708969 23639 ts_tablet_manager.cc:1403] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.000s
I20260709 13:06:48.709050 23645 ts_tablet_manager.cc:1403] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:06:48.712128 23639 raft_consensus.cc:359] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.713124 23639 raft_consensus.cc:385] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.714159 23645 raft_consensus.cc:359] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.715704 23645 raft_consensus.cc:385] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.716154 23645 raft_consensus.cc:740] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.717103 23638 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.717056 23645 consensus_queue.cc:260] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.717697 23638 ts_tablet_manager.cc:1403] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.090s	user 0.019s	sys 0.009s
I20260709 13:06:48.717921 23642 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.718547 23642 ts_tablet_manager.cc:1403] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.009s
I20260709 13:06:48.720657 23638 raft_consensus.cc:359] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.721421 23638 raft_consensus.cc:385] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.721823 23638 raft_consensus.cc:740] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.721438 23642 raft_consensus.cc:359] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.722469 23642 raft_consensus.cc:385] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.722827 23642 raft_consensus.cc:740] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.722594 23638 consensus_queue.cc:260] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.715667 23639 raft_consensus.cc:740] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.723536 23645 ts_tablet_manager.cc:1434] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:06:48.723728 23642 consensus_queue.cc:260] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.724602 23645 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.724231 23639 consensus_queue.cc:260] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.726063 23642 ts_tablet_manager.cc:1434] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:06:48.727059 23642 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.731060 23645 tablet_bootstrap.cc:654] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.733139 23642 tablet_bootstrap.cc:654] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.735308 23639 ts_tablet_manager.cc:1434] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:06:48.736824 23264 catalog_manager.cc:5697] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.738590 23661 raft_consensus.cc:493] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.739331 23661 raft_consensus.cc:515] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.740490 23639 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:48.742679 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.743988 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:48.744663 23368 raft_consensus.cc:2468] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:48.744871 23645 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.745458 23645 ts_tablet_manager.cc:1403] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.004s
I20260709 13:06:48.749064 23638 ts_tablet_manager.cc:1434] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:06:48.750166 23638 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.752239 23645 raft_consensus.cc:359] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.753188 23645 raft_consensus.cc:385] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.753342 23639 tablet_bootstrap.cc:654] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.753464 23645 raft_consensus.cc:740] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.757077 23713 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.754261 23645 consensus_queue.cc:260] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.757612 23713 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.756460 23484 leader_election.cc:304] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:48.759534 23771 raft_consensus.cc:2804] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.759950 23771 raft_consensus.cc:493] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.760497 23771 raft_consensus.cc:3060] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:48.760820 23482 leader_election.cc:343] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.763316 23645 ts_tablet_manager.cc:1434] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:06:48.763427 23707 consensus_queue.cc:1048] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.759857 23713 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:48.764443 23645 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.768160 23638 tablet_bootstrap.cc:654] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.768510 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:48.770969 23484 leader_election.cc:343] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.772389 23661 leader_election.cc:290] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:48.773530 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.777539 23736 consensus_queue.cc:1048] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.778189 23639 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:48.778748 23639 ts_tablet_manager.cc:1403] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.000s
I20260709 13:06:48.780401 23642 tablet_bootstrap.cc:492] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.780953 23642 ts_tablet_manager.cc:1403] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.003s
I20260709 13:06:48.783852 23645 tablet_bootstrap.cc:654] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.784226 23642 raft_consensus.cc:359] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.777649 23771 raft_consensus.cc:515] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.784943 23642 raft_consensus.cc:385] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.785255 23642 raft_consensus.cc:740] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.785971 23642 consensus_queue.cc:260] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.787987 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:48.788587 23642 ts_tablet_manager.cc:1434] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:48.788620 23368 raft_consensus.cc:3060] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.789186 23639 raft_consensus.cc:359] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.789754 23642 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.789933 23639 raft_consensus.cc:385] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.790273 23639 raft_consensus.cc:740] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
W20260709 13:06:48.791155 23482 leader_election.cc:343] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.791805 23482 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:48.793747 23707 raft_consensus.cc:2749] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.794226 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.797962 23707 consensus_queue.cc:1048] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:48.799108 23482 leader_election.cc:343] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.800966 23771 leader_election.cc:290] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:48.803544 23642 tablet_bootstrap.cc:654] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.800619 23368 raft_consensus.cc:2468] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.805177 23484 leader_election.cc:304] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:48.808923 23771 raft_consensus.cc:2804] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.810333 23771 raft_consensus.cc:697] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:48.822384 23771 consensus_queue.cc:237] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.825471 23642 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.826071 23642 ts_tablet_manager.cc:1403] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.000s
I20260709 13:06:48.827734 23638 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.829396 23642 raft_consensus.cc:359] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.830149 23642 raft_consensus.cc:385] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.830413 23642 raft_consensus.cc:740] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.830721 23736 consensus_queue.cc:1048] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.831120 23642 consensus_queue.cc:260] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.834329 23642 ts_tablet_manager.cc:1434] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:48.835299 23642 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.837422 23821 raft_consensus.cc:493] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.838011 23821 raft_consensus.cc:515] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.838999 23645 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.839568 23645 ts_tablet_manager.cc:1403] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.003s
I20260709 13:06:48.840461 23736 raft_consensus.cc:493] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.841025 23736 raft_consensus.cc:515] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:48.841220 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.842033 23443 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:48.843212 23736 leader_election.cc:290] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.843423 23482 leader_election.cc:304] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:48.844769 23713 raft_consensus.cc:2804] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.845148 23713 raft_consensus.cc:493] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.845487 23713 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.845973 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.846668 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.847345 23592 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
W20260709 13:06:48.848745 23333 leader_election.cc:343] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.828253 23638 ts_tablet_manager.cc:1403] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.078s	user 0.006s	sys 0.004s
I20260709 13:06:48.845847 23821 leader_election.cc:290] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.854250 23638 raft_consensus.cc:359] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.855087 23638 raft_consensus.cc:385] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.855598 23638 raft_consensus.cc:740] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.856208 23713 raft_consensus.cc:515] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.863596 23713 leader_election.cc:290] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:48.856294 23638 consensus_queue.cc:260] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.864760 23691 raft_consensus.cc:493] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.865394 23691 raft_consensus.cc:515] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.867419 23691 leader_election.cc:290] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.867702 23638 ts_tablet_manager.cc:1434] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:06:48.868752 23638 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.869248 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.863494 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:48.869961 23518 raft_consensus.cc:2468] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
W20260709 13:06:48.871342 23333 leader_election.cc:343] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.871886 23333 leader_election.cc:304] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:48.872160 23556 leader_election.cc:304] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:48.855058 23639 consensus_queue.cc:260] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.873039 23736 raft_consensus.cc:2749] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:48.873615 23772 raft_consensus.cc:493] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.873809 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.874254 23772 raft_consensus.cc:515] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.874835 23645 raft_consensus.cc:359] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.875547 23645 raft_consensus.cc:385] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.874536 23443 raft_consensus.cc:2468] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:48.876025 23645 raft_consensus.cc:740] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.876585 23772 leader_election.cc:290] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.877635 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:48.878340 23444 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:48.878157 23645 consensus_queue.cc:260] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.877648 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:48.879737 23591 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:48.880326 23645 ts_tablet_manager.cc:1434] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.000s
I20260709 13:06:48.881279 23333 leader_election.cc:304] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:48.882300 23772 raft_consensus.cc:2804] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.882666 23772 raft_consensus.cc:493] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.873688 23691 raft_consensus.cc:2804] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.883371 23691 raft_consensus.cc:493] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.883031 23772 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.883723 23691 raft_consensus.cc:3060] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.886886 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:48.887578 23444 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.889829 23645 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.894378 23772 raft_consensus.cc:515] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.901561 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.902417 23638 tablet_bootstrap.cc:654] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.909194 23591 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.896523 23444 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.913358 23482 leader_election.cc:304] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:48.903599 23645 tablet_bootstrap.cc:654] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.915808 23821 raft_consensus.cc:2804] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.916396 23821 raft_consensus.cc:697] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:48.917512 23821 consensus_queue.cc:237] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.905378 23707 consensus_queue.cc:1048] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.921782 23645 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:48.922394 23645 ts_tablet_manager.cc:1403] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.003s
I20260709 13:06:48.903898 23639 ts_tablet_manager.cc:1434] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.125s	user 0.006s	sys 0.001s
I20260709 13:06:48.924458 23639 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:48.926076 23638 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:48.926594 23638 ts_tablet_manager.cc:1403] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.001s
I20260709 13:06:48.907202 23642 tablet_bootstrap.cc:654] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.929203 23444 raft_consensus.cc:3060] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.929420 23638 raft_consensus.cc:359] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.930135 23638 raft_consensus.cc:385] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.930397 23638 raft_consensus.cc:740] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.931120 23638 consensus_queue.cc:260] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.933264 23638 ts_tablet_manager.cc:1434] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:06:48.934343 23638 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:48.931686 23691 raft_consensus.cc:515] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:48.939731 23264 catalog_manager.cc:5697] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:48.942003 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:48.942466 23691 leader_election.cc:290] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:48.942734 23519 raft_consensus.cc:3060] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.943307 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c0cd2ce17e48709b4a82c121539b62" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:48.943922 23444 raft_consensus.cc:3060] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.950146 23642 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:48.950749 23642 ts_tablet_manager.cc:1403] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 13:06:48.953750 23642 raft_consensus.cc:359] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.955096 23772 leader_election.cc:290] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:48.954495 23642 raft_consensus.cc:385] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.965145 23642 raft_consensus.cc:740] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.966974 23642 consensus_queue.cc:260] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:48.956148 23831 consensus_queue.cc:1048] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:48.968578 23639 tablet_bootstrap.cc:654] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.957170 23645 raft_consensus.cc:359] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.973179 23645 raft_consensus.cc:385] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.973608 23645 raft_consensus.cc:740] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.959199 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:48.960392 23638 tablet_bootstrap.cc:654] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.919888 23591 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:48.974802 23593 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.956511 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:48.977203 23443 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.978605 23642 ts_tablet_manager.cc:1434] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:06:48.978058 23645 consensus_queue.cc:260] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:48.981006 23645 ts_tablet_manager.cc:1434] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.000s
I20260709 13:06:48.984251 23519 raft_consensus.cc:2468] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:48.985859 23644 consensus_queue.cc:1048] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:48.987159 23444 raft_consensus.cc:2468] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:48.988183 23642 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:48.988961 23591 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.996999 23645 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:48.990788 23556 leader_election.cc:304] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:49.005555 23642 tablet_bootstrap.cc:654] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.007215 23691 raft_consensus.cc:2804] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.008069 23691 raft_consensus.cc:697] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:49.008981 23691 consensus_queue.cc:237] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.009568 23644 consensus_queue.cc:1048] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.002838 23638 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:49.013607 23638 ts_tablet_manager.cc:1403] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.079s	user 0.006s	sys 0.006s
I20260709 13:06:49.017833 23638 raft_consensus.cc:359] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.018555 23638 raft_consensus.cc:385] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.018814 23638 raft_consensus.cc:740] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.019449 23638 consensus_queue.cc:260] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.025626 23639 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:49.027379 23642 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.027890 23642 ts_tablet_manager.cc:1403] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.008s
I20260709 13:06:49.030725 23642 raft_consensus.cc:359] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.031458 23642 raft_consensus.cc:385] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.031713 23642 raft_consensus.cc:740] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.032655 23642 consensus_queue.cc:260] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.026372 23639 ts_tablet_manager.cc:1403] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.102s	user 0.010s	sys 0.004s
I20260709 13:06:49.034574 23642 ts_tablet_manager.cc:1434] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:49.035532 23642 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:49.039026 23265 catalog_manager.cc:5697] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.044054 23642 tablet_bootstrap.cc:654] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.045619 23638 ts_tablet_manager.cc:1434] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:06:49.046742 23638 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:49.051105 23443 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:49.051754 23710 consensus_queue.cc:1048] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.056268 23333 leader_election.cc:304] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.052582 23593 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:49.057163 23644 raft_consensus.cc:2804] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.057827 23644 raft_consensus.cc:697] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:49.058702 23644 consensus_queue.cc:237] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.052335 23264 catalog_manager.cc:5697] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.053366 23645 tablet_bootstrap.cc:654] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.052506 23638 tablet_bootstrap.cc:654] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.068034 23710 consensus_queue.cc:1048] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.085367 23645 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:49.085951 23645 ts_tablet_manager.cc:1403] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.000s
I20260709 13:06:49.089700 23645 raft_consensus.cc:359] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.090395 23645 raft_consensus.cc:385] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.090713 23645 raft_consensus.cc:740] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.095269 23642 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.095824 23642 ts_tablet_manager.cc:1403] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.002s
I20260709 13:06:49.097007 23645 consensus_queue.cc:260] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.098563 23642 raft_consensus.cc:359] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.099277 23642 raft_consensus.cc:385] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.099522 23642 raft_consensus.cc:740] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.099638 23645 ts_tablet_manager.cc:1434] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:06:49.100602 23645 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:49.100231 23642 consensus_queue.cc:260] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.102698 23642 ts_tablet_manager.cc:1434] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:49.102849 23638 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:49.103650 23642 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:49.108255 23645 tablet_bootstrap.cc:654] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.107851 23644 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.130955 23771 raft_consensus.cc:493] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.131498 23771 raft_consensus.cc:515] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.132331 23638 ts_tablet_manager.cc:1403] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.086s	user 0.012s	sys 0.003s
I20260709 13:06:49.131438 23644 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.134577 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.135113 23771 leader_election.cc:290] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:49.135308 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.135378 23443 raft_consensus.cc:2468] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:49.135725 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:49.136013 23445 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:49.137042 23482 leader_election.cc:304] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:49.137595 23333 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
W20260709 13:06:49.138036 23482 leader_election.cc:343] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.138545 23736 raft_consensus.cc:2804] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.138931 23736 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.118095 23639 raft_consensus.cc:359] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.139283 23736 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.139387 23639 raft_consensus.cc:385] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.140460 23639 raft_consensus.cc:740] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.141409 23642 tablet_bootstrap.cc:654] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.141314 23639 consensus_queue.cc:260] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.147596 23736 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.148252 23645 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:49.138952 23771 raft_consensus.cc:2804] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.149083 23771 raft_consensus.cc:493] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.149210 23645 ts_tablet_manager.cc:1403] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.024s
I20260709 13:06:49.149488 23771 raft_consensus.cc:3060] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.150590 23644 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.150769 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:49.152244 23639 ts_tablet_manager.cc:1434] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.119s	user 0.012s	sys 0.000s
W20260709 13:06:49.152335 23333 leader_election.cc:343] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.152338 23645 raft_consensus.cc:359] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.153791 23642 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.154956 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:49.159222 23645 raft_consensus.cc:385] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.159391 23445 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.159569 23645 raft_consensus.cc:740] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.159580 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:49.160553 23645 consensus_queue.cc:260] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
W20260709 13:06:49.161418 23333 leader_election.cc:343] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.162834 23645 ts_tablet_manager.cc:1434] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:06:49.163628 23736 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.166102 23638 raft_consensus.cc:359] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.167284 23638 raft_consensus.cc:385] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.167574 23638 raft_consensus.cc:740] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.170176 23645 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:49.158550 23771 raft_consensus.cc:515] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.156806 23639 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:49.173187 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:49.173823 23443 raft_consensus.cc:3060] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.174467 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:49.174686 23445 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
W20260709 13:06:49.176002 23482 leader_election.cc:343] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.175992 23333 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:49.176760 23645 tablet_bootstrap.cc:654] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.177562 23644 raft_consensus.cc:2804] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.178138 23644 raft_consensus.cc:697] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:49.179011 23644 consensus_queue.cc:237] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.181774 23443 raft_consensus.cc:2468] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:49.168283 23638 consensus_queue.cc:260] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.173336 23642 ts_tablet_manager.cc:1403] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.001s
I20260709 13:06:49.183393 23482 leader_election.cc:304] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:49.183876 23638 ts_tablet_manager.cc:1434] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.002s
I20260709 13:06:49.184258 23821 raft_consensus.cc:2804] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.184749 23821 raft_consensus.cc:697] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:49.185225 23771 leader_election.cc:290] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:49.185089 23642 raft_consensus.cc:359] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.184680 23265 catalog_manager.cc:5697] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.185897 23638 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:49.185757 23821 consensus_queue.cc:237] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.186432 23639 tablet_bootstrap.cc:654] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.192361 23638 tablet_bootstrap.cc:654] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.198313 23267 catalog_manager.cc:5697] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.185796 23642 raft_consensus.cc:385] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.200266 23639 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:49.200132 23642 raft_consensus.cc:740] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.200809 23639 ts_tablet_manager.cc:1403] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.000s
I20260709 13:06:49.201463 23642 consensus_queue.cc:260] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.205721 23642 ts_tablet_manager.cc:1434] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:06:49.208702 23642 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:49.207146 23639 raft_consensus.cc:359] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.211967 23639 raft_consensus.cc:385] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.212287 23639 raft_consensus.cc:740] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.213143 23639 consensus_queue.cc:260] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.215989 23639 ts_tablet_manager.cc:1434] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:06:49.217206 23639 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:49.221928 23645 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:49.223249 23645 ts_tablet_manager.cc:1403] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.001s
I20260709 13:06:49.216444 23644 raft_consensus.cc:493] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.225561 23638 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:49.226064 23638 ts_tablet_manager.cc:1403] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.012s
I20260709 13:06:49.226877 23645 raft_consensus.cc:359] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.227645 23645 raft_consensus.cc:385] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.227965 23645 raft_consensus.cc:740] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.228685 23645 consensus_queue.cc:260] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.223155 23642 tablet_bootstrap.cc:654] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.224608 23644 raft_consensus.cc:515] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.231348 23645 ts_tablet_manager.cc:1434] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:49.228878 23638 raft_consensus.cc:359] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.232371 23644 leader_election.cc:290] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.232455 23638 raft_consensus.cc:385] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.242527 23638 raft_consensus.cc:740] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.232620 23645 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:49.223716 23639 tablet_bootstrap.cc:654] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.234344 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.234726 23691 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.243743 23443 raft_consensus.cc:2468] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:49.243911 23691 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.245298 23638 consensus_queue.cc:260] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.247382 23638 ts_tablet_manager.cc:1434] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:06:49.248275 23638 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:49.249607 23333 leader_election.cc:304] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.251675 23639 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:49.252250 23831 consensus_queue.cc:1048] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.254078 23638 tablet_bootstrap.cc:654] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.254202 23645 tablet_bootstrap.cc:654] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.255836 23772 raft_consensus.cc:2804] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.256299 23772 raft_consensus.cc:493] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.256613 23772 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.258618 23691 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.259378 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.260021 23368 raft_consensus.cc:2468] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:49.261188 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.252279 23639 ts_tablet_manager.cc:1403] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.004s
I20260709 13:06:49.262809 23558 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
W20260709 13:06:49.264228 23556 leader_election.cc:343] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.265287 23691 raft_consensus.cc:2804] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.265791 23691 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.268121 23691 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.267175 23772 raft_consensus.cc:515] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.269057 23639 raft_consensus.cc:359] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.269776 23639 raft_consensus.cc:385] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.270097 23639 raft_consensus.cc:740] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.235589 23642 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.236091 23736 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.236364 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:49.274782 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:49.275135 23591 raft_consensus.cc:2468] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:49.274821 23736 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.274992 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:49.277240 23443 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.278235 23736 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:49.279414 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:49.280037 23517 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:49.281509 23333 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.282482 23736 raft_consensus.cc:2804] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.282868 23736 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.283212 23736 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.285040 23677 raft_consensus.cc:493] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.285702 23677 raft_consensus.cc:515] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.286293 23642 ts_tablet_manager.cc:1403] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.001s
I20260709 13:06:49.286904 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:49.287935 23677 leader_election.cc:290] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.285153 23724 raft_consensus.cc:493] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.289288 23724 raft_consensus.cc:515] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.289508 23639 consensus_queue.cc:260] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.290302 23831 raft_consensus.cc:493] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.290539 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.290768 23831 raft_consensus.cc:515] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.291549 23724 leader_election.cc:290] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:49.291510 23369 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:49.293427 23410 leader_election.cc:304] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.294446 23677 raft_consensus.cc:2804] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.287873 23593 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:49.294780 23771 raft_consensus.cc:493] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.295004 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:49.294821 23677 raft_consensus.cc:493] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.295506 23771 raft_consensus.cc:515] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.296054 23677 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.296769 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:49.297459 23517 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:49.297936 23771 leader_election.cc:290] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.305464 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.305992 23831 leader_election.cc:290] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:49.307163 23443 raft_consensus.cc:2468] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:49.307550 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:49.308307 23591 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:49.309811 23333 leader_election.cc:304] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.310849 23728 raft_consensus.cc:2804] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.314366 23639 ts_tablet_manager.cc:1434] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 13:06:49.317910 23639 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:49.323129 23593 raft_consensus.cc:3060] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.326579 23642 raft_consensus.cc:359] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.327703 23642 raft_consensus.cc:385] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.328061 23642 raft_consensus.cc:740] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.274005 23772 leader_election.cc:290] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.332991 23642 consensus_queue.cc:260] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.350782 23639 tablet_bootstrap.cc:654] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.349800 23266 catalog_manager.cc:5697] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.338474 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.333789 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff658c154728474ab62925bcfa9e4fae" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.357110 23728 raft_consensus.cc:697] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:49.343371 23482 leader_election.cc:304] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:49.369470 23642 ts_tablet_manager.cc:1434] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.002s
I20260709 13:06:49.371272 23642 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:49.344277 23691 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.377061 23558 leader_election.cc:304] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:49.375414 23821 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.378703 23821 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.383850 23728 consensus_queue.cc:237] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.368214 23368 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:49.388310 23677 raft_consensus.cc:515] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.389382 23831 raft_consensus.cc:2804] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.389912 23831 raft_consensus.cc:493] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.390323 23831 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.382730 23821 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.389546 23445 raft_consensus.cc:2393] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:49.394492 23691 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:49.397532 23724 raft_consensus.cc:2749] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.398365 23821 consensus_queue.cc:1048] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.380717 23690 raft_consensus.cc:493] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.401216 23690 raft_consensus.cc:515] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.402958 23736 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.403870 23677 leader_election.cc:290] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.406920 23638 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:49.407603 23638 ts_tablet_manager.cc:1403] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.160s	user 0.014s	sys 0.004s
I20260709 13:06:49.411670 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.414119 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:49.417563 23517 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.422250 23690 leader_election.cc:290] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.422767 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:49.424141 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:49.423570 23593 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.425012 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:49.425889 23444 raft_consensus.cc:2393] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:49.426563 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:49.427205 23519 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.430858 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "671e4639fc5945aba1d87c67ed1fc249" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:49.432696 23556 leader_election.cc:343] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.434074 23484 leader_election.cc:304] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:49.436084 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:49.436760 23367 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:49.442291 23408 leader_election.cc:343] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.446132 23831 raft_consensus.cc:515] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.447141 23736 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:49.459995 23645 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:49.460726 23645 ts_tablet_manager.cc:1403] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.228s	user 0.050s	sys 0.045s
I20260709 13:06:49.475368 23866 consensus_queue.cc:1048] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.480912 23736 consensus_queue.cc:1048] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.487246 23593 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:49.488664 23691 consensus_queue.cc:1048] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:49.489370 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:49.489136 23645 raft_consensus.cc:359] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.489897 23645 raft_consensus.cc:385] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.489812 23519 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:49.490252 23645 raft_consensus.cc:740] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.490379 23821 raft_consensus.cc:2764] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:49.492291 23333 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.492136 23645 consensus_queue.cc:260] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.492951 23367 raft_consensus.cc:2468] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:49.493718 23408 leader_election.cc:304] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:49.494202 23558 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:49.495162 23865 raft_consensus.cc:2804] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.495466 23691 raft_consensus.cc:2804] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.495730 23865 raft_consensus.cc:697] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:49.495995 23691 raft_consensus.cc:697] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:49.497021 23642 tablet_bootstrap.cc:654] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.496738 23865 consensus_queue.cc:237] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.496927 23691 consensus_queue.cc:237] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.522473 23831 leader_election.cc:290] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:49.527200 23517 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:49.534206 23645 ts_tablet_manager.cc:1434] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 13:06:49.535750 23645 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:49.537585 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:49.538504 23367 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.547093 23831 consensus_queue.cc:1048] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.530211 23728 raft_consensus.cc:2804] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.561856 23638 raft_consensus.cc:359] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.565804 23638 raft_consensus.cc:385] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.566363 23638 raft_consensus.cc:740] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.567112 23638 consensus_queue.cc:260] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.577637 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.578438 23369 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:49.593560 23484 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:49.592489 23367 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:49.605952 23638 ts_tablet_manager.cc:1434] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.198s	user 0.008s	sys 0.000s
I20260709 13:06:49.607510 23638 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:49.599947 23728 raft_consensus.cc:697] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:49.618853 23728 consensus_queue.cc:237] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.629273 23677 raft_consensus.cc:493] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.630012 23677 raft_consensus.cc:515] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.635478 23677 leader_election.cc:290] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.635684 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.637331 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:49.636511 23369 raft_consensus.cc:2468] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
W20260709 13:06:49.638994 23408 leader_election.cc:343] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.597869 23831 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.640472 23410 leader_election.cc:304] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:49.642143 23677 raft_consensus.cc:2804] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.642717 23677 raft_consensus.cc:493] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.643226 23677 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.594480 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:49.652348 23591 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.576560 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "671e4639fc5945aba1d87c67ed1fc249" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:49.657577 23368 raft_consensus.cc:2468] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:49.659965 23410 leader_election.cc:304] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:49.661511 23710 raft_consensus.cc:2804] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.662075 23710 raft_consensus.cc:493] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.662559 23710 raft_consensus.cc:3060] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.679571 23772 consensus_queue.cc:1048] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.682474 23713 consensus_queue.cc:1048] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.687453 23645 tablet_bootstrap.cc:654] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.687202 23677 raft_consensus.cc:515] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.689364 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:49.690256 23367 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.696436 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:49.697135 23370 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.705066 23370 raft_consensus.cc:2468] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:49.706405 23410 leader_election.cc:304] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:49.707965 23690 raft_consensus.cc:2804] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.707850 23772 consensus_queue.cc:1048] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.713135 23677 leader_election.cc:290] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.714713 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:49.716360 23408 leader_election.cc:343] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.716888 23831 raft_consensus.cc:2749] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.758790 23639 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:49.759637 23639 ts_tablet_manager.cc:1403] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.444s	user 0.020s	sys 0.048s
I20260709 13:06:49.776664 23591 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:49.777010 23642 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.777647 23642 ts_tablet_manager.cc:1403] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.407s	user 0.018s	sys 0.004s
I20260709 13:06:49.780530 23642 raft_consensus.cc:359] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.781021 23638 tablet_bootstrap.cc:654] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.781284 23642 raft_consensus.cc:385] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.781597 23642 raft_consensus.cc:740] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.782370 23642 consensus_queue.cc:260] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.784031 23639 raft_consensus.cc:359] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.784996 23639 raft_consensus.cc:385] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.785342 23639 raft_consensus.cc:740] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.786269 23639 consensus_queue.cc:260] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.790313 23642 ts_tablet_manager.cc:1434] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:49.791442 23642 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:49.812024 23642 tablet_bootstrap.cc:654] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.841895 23639 ts_tablet_manager.cc:1434] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.002s
I20260709 13:06:49.842757 23367 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:49.843355 23482 leader_election.cc:304] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:49.852082 23639 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:49.833727 23690 raft_consensus.cc:697] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:49.851740 23831 raft_consensus.cc:2804] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.866237 23831 raft_consensus.cc:697] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:49.875671 23638 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:49.876544 23638 ts_tablet_manager.cc:1403] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.269s	user 0.018s	sys 0.023s
I20260709 13:06:49.861999 23710 raft_consensus.cc:515] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.876363 23831 consensus_queue.cc:237] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.882373 23690 consensus_queue.cc:237] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:49.902215 23645 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:49.902748 23645 ts_tablet_manager.cc:1403] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.367s	user 0.017s	sys 0.020s
I20260709 13:06:49.899156 23724 raft_consensus.cc:493] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:49.913323 23876 log.cc:927] Time spent T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:06:49.935173 23638 raft_consensus.cc:359] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.935935 23638 raft_consensus.cc:385] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.936192 23638 raft_consensus.cc:740] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.937314 23638 consensus_queue.cc:260] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:49.911314 23724 raft_consensus.cc:515] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:49.941386 23642 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:49.944378 23642 ts_tablet_manager.cc:1403] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.153s	user 0.008s	sys 0.009s
I20260709 13:06:49.948572 23639 tablet_bootstrap.cc:654] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.948563 23642 raft_consensus.cc:359] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.949535 23642 raft_consensus.cc:385] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.949841 23642 raft_consensus.cc:740] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.953609 23645 raft_consensus.cc:359] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.954550 23645 raft_consensus.cc:385] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.954872 23645 raft_consensus.cc:740] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
W20260709 13:06:49.928057 23482 consensus_peers.cc:597] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 -> Peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Couldn't send request to peer ceb56da8fd5145648a898f5487219c6e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:49.957554 23638 ts_tablet_manager.cc:1434] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:06:49.959846 23638 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:49.956442 23645 consensus_queue.cc:260] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.955901 23642 consensus_queue.cc:260] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:49.963114 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "671e4639fc5945aba1d87c67ed1fc249" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:49.963974 23367 raft_consensus.cc:3060] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.965622 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "671e4639fc5945aba1d87c67ed1fc249" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:49.966274 23519 raft_consensus.cc:3060] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.964380 23881 consensus_queue.cc:1048] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:06:49.957774 23266 catalog_manager.cc:5697] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.973816 23866 consensus_queue.cc:1048] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:49.974252 23886 log.cc:927] Time spent T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:06:49.984264 23442 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.985309 23724 leader_election.cc:290] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:49.998198 23642 ts_tablet_manager.cc:1434] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
W20260709 13:06:49.960245 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 47082.6us
I20260709 13:06:50.005496 23642 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:50.013139 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.016331 23710 leader_election.cc:290] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:50.005226 23890 consensus_queue.cc:1048] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.072s
I20260709 13:06:50.054028 23645 ts_tablet_manager.cc:1434] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.151s	user 0.026s	sys 0.030s
I20260709 13:06:50.055652 23645 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:50.063062 23639 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:50.063742 23639 ts_tablet_manager.cc:1403] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.213s	user 0.030s	sys 0.056s
I20260709 13:06:50.064040 23638 tablet_bootstrap.cc:654] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:50.066069 23556 leader_election.cc:343] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.067724 23639 raft_consensus.cc:359] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.068802 23639 raft_consensus.cc:385] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.069167 23639 raft_consensus.cc:740] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.070092 23639 consensus_queue.cc:260] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.073237 23639 ts_tablet_manager.cc:1434] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:06:50.074492 23639 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:50.074317 23367 raft_consensus.cc:2468] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
W20260709 13:06:50.081597 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 54807.3us
I20260709 13:06:50.055140 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:50.092792 23710 consensus_queue.cc:1048] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:50.104629 23519 raft_consensus.cc:2468] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:50.112497 23864 consensus_queue.cc:1048] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.114538 23639 tablet_bootstrap.cc:654] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.124078 23410 leader_election.cc:304] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:50.125501 23710 raft_consensus.cc:2804] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.126020 23710 raft_consensus.cc:697] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:50.126856 23710 consensus_queue.cc:237] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.129678 23771 consensus_queue.cc:1048] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.135524 23592 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:50.140060 23556 leader_election.cc:343] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.132632 23724 raft_consensus.cc:493] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.143395 23724 raft_consensus.cc:515] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.145588 23724 leader_election.cc:290] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:50.140594 23556 leader_election.cc:304] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:50.147248 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2a928268b4d73afa32a47bbbcbd64" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:50.148744 23558 leader_election.cc:343] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.166090 23881 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.174338 23881 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.156186 23724 raft_consensus.cc:2749] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.157632 23728 consensus_queue.cc:1048] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:06:50.189028 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2a928268b4d73afa32a47bbbcbd64" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:50.189721 23444 raft_consensus.cc:2468] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:50.207599 23645 tablet_bootstrap.cc:654] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.224907 23642 tablet_bootstrap.cc:654] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.239529 23639 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:50.240190 23639 ts_tablet_manager.cc:1403] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.166s	user 0.003s	sys 0.009s
I20260709 13:06:50.233219 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.238381 23881 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:50.288959 23638 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:50.289708 23638 ts_tablet_manager.cc:1403] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.330s	user 0.033s	sys 0.035s
I20260709 13:06:50.296427 23639 raft_consensus.cc:359] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.297405 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:50.299988 23639 raft_consensus.cc:385] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.300364 23639 raft_consensus.cc:740] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.309494 23408 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:50.258330 23556 leader_election.cc:304] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:50.312649 23881 raft_consensus.cc:2749] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:50.321983 23866 raft_consensus.cc:2804] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.322676 23866 raft_consensus.cc:493] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.323163 23866 raft_consensus.cc:3060] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:50.325402 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 73477.9us
I20260709 13:06:50.328851 23691 consensus_queue.cc:1048] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.325340 23639 consensus_queue.cc:260] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.345161 23894 consensus_queue.cc:1048] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.346653 23736 consensus_queue.cc:1048] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.347559 23728 raft_consensus.cc:493] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.348059 23638 raft_consensus.cc:359] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.348140 23728 raft_consensus.cc:515] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.350792 23728 leader_election.cc:290] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.352064 23638 raft_consensus.cc:385] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.352433 23638 raft_consensus.cc:740] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.353181 23638 consensus_queue.cc:260] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.358553 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.358508 23866 raft_consensus.cc:515] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.354198 23264 catalog_manager.cc:5697] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.364308 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2a928268b4d73afa32a47bbbcbd64" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
W20260709 13:06:50.365276 23333 leader_election.cc:343] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.368445 23638 ts_tablet_manager.cc:1434] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 13:06:50.373270 23639 ts_tablet_manager.cc:1434] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.133s	user 0.008s	sys 0.004s
I20260709 13:06:50.374557 23639 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:50.362600 23866 leader_election.cc:290] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:50.375296 23638 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:50.365253 23367 raft_consensus.cc:3060] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.383599 23881 consensus_queue.cc:1048] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.385360 23367 raft_consensus.cc:2468] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:50.393994 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2a928268b4d73afa32a47bbbcbd64" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:50.394677 23442 raft_consensus.cc:3060] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.383069 23264 catalog_manager.cc:5697] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.401608 23558 leader_election.cc:304] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:50.405637 23866 raft_consensus.cc:2804] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.407011 23866 raft_consensus.cc:697] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:50.409853 23642 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:50.409376 23866 consensus_queue.cc:237] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.410368 23642 ts_tablet_manager.cc:1403] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.411s	user 0.025s	sys 0.011s
I20260709 13:06:50.410910 23645 tablet_bootstrap.cc:492] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:50.411427 23645 ts_tablet_manager.cc:1403] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.356s	user 0.013s	sys 0.006s
I20260709 13:06:50.416147 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.416451 23642 raft_consensus.cc:359] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.417075 23831 consensus_queue.cc:1048] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.420248 23642 raft_consensus.cc:385] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.420614 23642 raft_consensus.cc:740] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
W20260709 13:06:50.417681 23333 leader_election.cc:343] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.421458 23333 leader_election.cc:304] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:50.421845 23642 consensus_queue.cc:260] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.422923 23645 raft_consensus.cc:359] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.423959 23645 raft_consensus.cc:385] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.424248 23645 raft_consensus.cc:740] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.428303 23645 consensus_queue.cc:260] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.429188 23642 ts_tablet_manager.cc:1434] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:06:50.432965 23642 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:50.440174 23645 ts_tablet_manager.cc:1434] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:06:50.442968 23639 tablet_bootstrap.cc:654] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.449891 23645 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:50.452720 23863 raft_consensus.cc:2749] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.455224 23519 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.463104 23645 tablet_bootstrap.cc:654] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.464135 23442 raft_consensus.cc:2468] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:50.468820 23885 consensus_queue.cc:1048] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.476866 23831 raft_consensus.cc:493] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.477584 23831 raft_consensus.cc:515] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.482307 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.480517 23831 leader_election.cc:290] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.489795 23588 raft_consensus.cc:2468] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:50.491097 23638 tablet_bootstrap.cc:654] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.492723 23482 leader_election.cc:304] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:50.495065 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
W20260709 13:06:50.502555 23482 leader_election.cc:343] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.497881 23885 raft_consensus.cc:2804] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.511713 23885 raft_consensus.cc:493] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.512403 23885 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.504336 23881 consensus_queue.cc:1048] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:06:50.529804 23642 tablet_bootstrap.cc:654] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.536515 23866 raft_consensus.cc:493] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.537393 23866 raft_consensus.cc:515] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.545596 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:50.549800 23866 leader_election.cc:290] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:50.549954 23865 raft_consensus.cc:493] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.550679 23865 raft_consensus.cc:515] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.551301 23885 raft_consensus.cc:515] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.553210 23865 leader_election.cc:290] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.557273 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21203c7d4b1148e1a4e659daf1df428c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.558925 23645 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:50.559491 23645 ts_tablet_manager.cc:1403] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.023s
I20260709 13:06:50.559942 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.567054 23556 leader_election.cc:304] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:50.566910 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.567831 23866 raft_consensus.cc:2749] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.568751 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
W20260709 13:06:50.565855 23408 leader_election.cc:343] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.569890 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
W20260709 13:06:50.570466 23408 leader_election.cc:343] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.570601 23593 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:50.570380 23482 leader_election.cc:343] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.571030 23408 leader_election.cc:304] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:50.572479 23865 raft_consensus.cc:2749] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.575223 23645 raft_consensus.cc:359] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.576001 23645 raft_consensus.cc:385] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.576268 23645 raft_consensus.cc:740] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.576929 23638 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:50.577607 23638 ts_tablet_manager.cc:1403] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.205s	user 0.018s	sys 0.001s
I20260709 13:06:50.580125 23642 tablet_bootstrap.cc:492] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:50.580726 23642 ts_tablet_manager.cc:1403] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.149s	user 0.015s	sys 0.003s
I20260709 13:06:50.583225 23645 consensus_queue.cc:260] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.586110 23642 raft_consensus.cc:359] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.587986 23642 raft_consensus.cc:385] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.588374 23642 raft_consensus.cc:740] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.589043 23639 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:50.589632 23639 ts_tablet_manager.cc:1403] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.215s	user 0.028s	sys 0.014s
I20260709 13:06:50.586666 23645 ts_tablet_manager.cc:1434] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:06:50.589238 23642 consensus_queue.cc:260] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.590799 23645 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:50.591948 23642 ts_tablet_manager.cc:1434] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:06:50.564672 23885 leader_election.cc:290] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.597916 23642 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:50.609795 23642 tablet_bootstrap.cc:654] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.613765 23267 catalog_manager.cc:5697] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.615792 23267 catalog_manager.cc:5697] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.620951 23593 raft_consensus.cc:2468] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:50.622319 23482 leader_election.cc:304] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:50.624799 23638 raft_consensus.cc:359] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.625523 23638 raft_consensus.cc:385] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.625713 23638 raft_consensus.cc:740] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.626688 23639 raft_consensus.cc:359] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.627332 23639 raft_consensus.cc:385] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.627578 23639 raft_consensus.cc:740] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.628095 23639 consensus_queue.cc:260] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.626211 23638 consensus_queue.cc:260] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.638619 23885 raft_consensus.cc:2804] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.639176 23866 raft_consensus.cc:493] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.640002 23866 raft_consensus.cc:515] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.642069 23264 catalog_manager.cc:5697] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.644539 23866 leader_election.cc:290] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:50.646608 23639 ts_tablet_manager.cc:1434] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.002s
I20260709 13:06:50.643832 23264 catalog_manager.cc:5697] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } }
I20260709 13:06:50.647387 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:50.647709 23639 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:50.648119 23367 raft_consensus.cc:2468] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:50.649596 23558 leader_election.cc:304] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:50.649822 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.650641 23866 raft_consensus.cc:2804] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.651141 23866 raft_consensus.cc:493] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:50.651306 23556 leader_election.cc:343] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.651633 23866 raft_consensus.cc:3060] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.639256 23885 raft_consensus.cc:697] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:50.653196 23885 consensus_queue.cc:237] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.667865 23638 ts_tablet_manager.cc:1434] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.000s
I20260709 13:06:50.656628 23645 tablet_bootstrap.cc:654] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.670158 23642 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:50.677915 23642 ts_tablet_manager.cc:1403] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.000s
I20260709 13:06:50.680351 23638 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:50.681566 23642 raft_consensus.cc:359] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.683159 23642 raft_consensus.cc:385] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.683516 23642 raft_consensus.cc:740] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.679634 23866 raft_consensus.cc:515] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.692809 23831 raft_consensus.cc:493] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.693475 23831 raft_consensus.cc:515] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.695719 23831 leader_election.cc:290] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.690038 23642 consensus_queue.cc:260] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.698264 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:50.698738 23642 ts_tablet_manager.cc:1434] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:06:50.699050 23445 raft_consensus.cc:2468] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:50.700059 23642 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:50.701509 23482 leader_election.cc:304] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:50.704475 23821 raft_consensus.cc:2804] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.709385 23821 raft_consensus.cc:493] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.709821 23821 raft_consensus.cc:3060] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.712411 23865 consensus_queue.cc:1048] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:50.712993 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:50.721237 23881 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.721999 23881 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.726466 23265 catalog_manager.cc:5697] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.729466 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.728204 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:50.733551 23410 leader_election.cc:343] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:50.734447 23408 leader_election.cc:343] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.734885 23408 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:50.735620 23710 raft_consensus.cc:2749] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.724808 23881 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:50.742856 23865 consensus_queue.cc:1048] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.751399 23645 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:50.752058 23645 ts_tablet_manager.cc:1403] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.017s
I20260709 13:06:50.752969 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:50.754604 23556 leader_election.cc:343] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.754969 23645 raft_consensus.cc:359] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.757714 23645 raft_consensus.cc:385] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.758009 23645 raft_consensus.cc:740] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.758687 23645 consensus_queue.cc:260] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.762827 23645 ts_tablet_manager.cc:1434] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:06:50.763825 23645 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:50.768293 23642 tablet_bootstrap.cc:654] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.708750 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.773743 23638 tablet_bootstrap.cc:654] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:50.774197 23482 leader_election.cc:343] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.777535 23645 tablet_bootstrap.cc:654] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.659848 23639 tablet_bootstrap.cc:654] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.768823 23866 leader_election.cc:290] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:50.769126 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:50.780632 23368 raft_consensus.cc:3060] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.785404 23266 catalog_manager.cc:5697] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } }
I20260709 13:06:50.800652 23639 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:50.801265 23639 ts_tablet_manager.cc:1403] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.154s	user 0.011s	sys 0.002s
I20260709 13:06:50.804095 23639 raft_consensus.cc:359] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.804816 23639 raft_consensus.cc:385] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.805153 23639 raft_consensus.cc:740] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.805971 23639 consensus_queue.cc:260] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.812903 23639 ts_tablet_manager.cc:1434] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:06:50.813880 23639 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:50.814524 23866 consensus_queue.cc:1048] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.771381 23821 raft_consensus.cc:515] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.819537 23368 raft_consensus.cc:2468] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:50.823119 23558 leader_election.cc:304] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:50.826537 23639 tablet_bootstrap.cc:654] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.827082 23913 raft_consensus.cc:2804] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.828080 23913 raft_consensus.cc:697] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:50.829622 23691 consensus_queue.cc:1048] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.830958 23642 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:50.831468 23642 ts_tablet_manager.cc:1403] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.003s
I20260709 13:06:50.828999 23913 consensus_queue.cc:237] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.834767 23638 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:50.835259 23638 ts_tablet_manager.cc:1403] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.000s
I20260709 13:06:50.839514 23642 raft_consensus.cc:359] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.840263 23642 raft_consensus.cc:385] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.840519 23642 raft_consensus.cc:740] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.845688 23645 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:50.846360 23645 ts_tablet_manager.cc:1403] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.020s
I20260709 13:06:50.845670 23642 consensus_queue.cc:260] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.853581 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:50.854914 23638 raft_consensus.cc:359] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.859671 23638 raft_consensus.cc:385] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.860067 23638 raft_consensus.cc:740] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
W20260709 13:06:50.855937 23482 leader_election.cc:343] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.857167 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:50.864214 23442 raft_consensus.cc:3060] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.866035 23645 raft_consensus.cc:359] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.866953 23645 raft_consensus.cc:385] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.867257 23645 raft_consensus.cc:740] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.872509 23691 raft_consensus.cc:493] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.873237 23691 raft_consensus.cc:515] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.874740 23881 raft_consensus.cc:493] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.875276 23881 raft_consensus.cc:515] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.877756 23881 leader_election.cc:290] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.880771 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:50.881368 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.881872 23367 raft_consensus.cc:2468] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:50.883872 23558 leader_election.cc:304] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:50.877789 23593 raft_consensus.cc:3060] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:50.885042 23556 leader_election.cc:343] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.860941 23638 consensus_queue.cc:260] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:50.885849 23895 raft_consensus.cc:2804] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.906714 23895 raft_consensus.cc:493] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.907306 23895 raft_consensus.cc:3060] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.906019 23645 consensus_queue.cc:260] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.886425 23691 leader_election.cc:290] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:50.888118 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.888824 23821 leader_election.cc:290] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:50.891175 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:50.927299 23592 raft_consensus.cc:2393] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:50.929124 23408 leader_election.cc:304] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:50.930012 23881 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.941866 23881 raft_consensus.cc:2749] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.942281 23638 ts_tablet_manager.cc:1434] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.107s	user 0.010s	sys 0.000s
I20260709 13:06:50.953835 23442 raft_consensus.cc:2468] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:50.945122 23642 ts_tablet_manager.cc:1434] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.001s
I20260709 13:06:50.955386 23482 leader_election.cc:304] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e
W20260709 13:06:50.949293 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 50079.5us
I20260709 13:06:50.956039 23642 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:50.956154 23821 raft_consensus.cc:2804] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.956877 23638 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:50.953365 23639 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:50.956751 23821 raft_consensus.cc:697] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:50.958498 23821 consensus_queue.cc:237] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.960392 23639 ts_tablet_manager.cc:1403] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.147s	user 0.029s	sys 0.053s
I20260709 13:06:50.963490 23638 tablet_bootstrap.cc:654] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.948212 23895 raft_consensus.cc:515] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.955538 23645 ts_tablet_manager.cc:1434] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.002s
I20260709 13:06:50.965924 23645 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:50.966305 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:50.967789 23556 leader_election.cc:343] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.971360 23945 raft_consensus.cc:493] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.971895 23945 raft_consensus.cc:515] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:50.972437 23267 catalog_manager.cc:5697] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.975924 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:50.977828 23639 raft_consensus.cc:359] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.978679 23639 raft_consensus.cc:385] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.978927 23639 raft_consensus.cc:740] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.979152 23638 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:50.979633 23638 ts_tablet_manager.cc:1403] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.007s
I20260709 13:06:50.979632 23639 consensus_queue.cc:260] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:50.981904 23639 ts_tablet_manager.cc:1434] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:06:50.982316 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:50.982829 23639 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:50.982470 23638 raft_consensus.cc:359] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:50.983202 23638 raft_consensus.cc:385] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.983482 23638 raft_consensus.cc:740] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
W20260709 13:06:50.984077 23556 leader_election.cc:343] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.984237 23638 consensus_queue.cc:260] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:50.984906 23556 leader_election.cc:343] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.985461 23556 leader_election.cc:304] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:50.986344 23945 leader_election.cc:290] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:50.986439 23866 raft_consensus.cc:2749] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.986908 23638 ts_tablet_manager.cc:1434] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:50.974090 23645 tablet_bootstrap.cc:654] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.987942 23638 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:50.993306 23639 tablet_bootstrap.cc:654] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.994247 23638 tablet_bootstrap.cc:654] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.995366 23895 leader_election.cc:290] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:50.997256 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:50.998085 23368 raft_consensus.cc:3060] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.998587 23642 tablet_bootstrap.cc:654] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.025992 23642 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:51.026487 23642 ts_tablet_manager.cc:1403] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.004s
I20260709 13:06:51.028014 23736 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.030644 23736 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.033502 23736 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.028589 23638 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.034634 23638 ts_tablet_manager.cc:1403] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.009s
I20260709 13:06:51.034051 23368 raft_consensus.cc:2468] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:51.036659 23558 leader_election.cc:304] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:51.037063 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.037314 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.034027 23642 raft_consensus.cc:359] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.038300 23642 raft_consensus.cc:385] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.038479 23895 raft_consensus.cc:2804] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.038671 23642 raft_consensus.cc:740] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.039043 23895 raft_consensus.cc:697] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:51.039747 23642 consensus_queue.cc:260] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.039968 23895 consensus_queue.cc:237] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
W20260709 13:06:51.043936 23333 leader_election.cc:343] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:51.046092 23333 leader_election.cc:343] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.046625 23333 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:51.048640 23728 raft_consensus.cc:2749] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.050348 23642 ts_tablet_manager.cc:1434] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:06:51.055738 23642 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:51.062705 23645 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.063328 23645 ts_tablet_manager.cc:1403] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.021s
I20260709 13:06:51.066567 23645 raft_consensus.cc:359] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.067238 23645 raft_consensus.cc:385] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.067413 23642 tablet_bootstrap.cc:654] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.067530 23645 raft_consensus.cc:740] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.068684 23645 consensus_queue.cc:260] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.072259 23645 ts_tablet_manager.cc:1434] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:51.073241 23645 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:51.075239 23268 catalog_manager.cc:5697] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.079506 23645 tablet_bootstrap.cc:654] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.080752 23710 raft_consensus.cc:493] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.080859 23713 consensus_queue.cc:1048] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.081398 23710 raft_consensus.cc:515] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.083670 23710 leader_election.cc:290] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.084563 23831 raft_consensus.cc:493] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.085551 23831 raft_consensus.cc:515] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.092913 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.095167 23370 raft_consensus.cc:2393] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:51.097221 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7ef8cd1ea7446f6bd4a4f133cbf701c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:51.097926 23517 raft_consensus.cc:2468] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:51.050642 23638 raft_consensus.cc:359] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.101974 23638 raft_consensus.cc:385] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.102326 23638 raft_consensus.cc:740] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.088694 23831 leader_election.cc:290] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:51.105545 23408 leader_election.cc:304] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:51.116991 23881 raft_consensus.cc:2804] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.117375 23881 raft_consensus.cc:493] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.117691 23881 raft_consensus.cc:3060] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.117920 23639 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:51.118436 23639 ts_tablet_manager.cc:1403] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.040s
I20260709 13:06:51.112179 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1d7ecd2e941d289aefe4228c0e673" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.120643 23266 catalog_manager.cc:5697] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.126420 23639 raft_consensus.cc:359] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.127463 23639 raft_consensus.cc:385] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.127815 23639 raft_consensus.cc:740] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.124006 23638 consensus_queue.cc:260] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.105391 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7ef8cd1ea7446f6bd4a4f133cbf701c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.137413 23638 ts_tablet_manager.cc:1434] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 13:06:51.138875 23638 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.141831 23639 consensus_queue.cc:260] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:51.140432 23410 leader_election.cc:343] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.143676 23482 leader_election.cc:304] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.148033 23831 raft_consensus.cc:3060] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.149544 23645 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.150077 23645 ts_tablet_manager.cc:1403] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.013s
I20260709 13:06:51.160653 23831 raft_consensus.cc:2749] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.161975 23881 raft_consensus.cc:515] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.164163 23881 leader_election.cc:290] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.164904 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7ef8cd1ea7446f6bd4a4f133cbf701c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:51.165235 23639 ts_tablet_manager.cc:1434] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:06:51.166298 23639 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
W20260709 13:06:51.166592 23410 leader_election.cc:343] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.168001 23645 raft_consensus.cc:359] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.169822 23645 raft_consensus.cc:385] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.170152 23645 raft_consensus.cc:740] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.172968 23645 consensus_queue.cc:260] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.178975 23642 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:51.178804 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7ef8cd1ea7446f6bd4a4f133cbf701c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:51.179551 23642 ts_tablet_manager.cc:1403] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.124s	user 0.014s	sys 0.015s
I20260709 13:06:51.179556 23519 raft_consensus.cc:3060] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.183827 23642 raft_consensus.cc:359] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.184569 23642 raft_consensus.cc:385] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.184831 23642 raft_consensus.cc:740] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.185497 23642 consensus_queue.cc:260] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.187198 23519 raft_consensus.cc:2468] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:51.187626 23642 ts_tablet_manager.cc:1434] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:51.199337 23642 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:51.201570 23821 consensus_queue.cc:1048] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.205412 23642 tablet_bootstrap.cc:654] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.210719 23408 leader_election.cc:304] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.212736 23639 tablet_bootstrap.cc:654] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.213104 23645 ts_tablet_manager.cc:1434] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:06:51.214447 23865 raft_consensus.cc:2804] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.215037 23865 raft_consensus.cc:697] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:51.195111 23638 tablet_bootstrap.cc:654] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.215997 23865 consensus_queue.cc:237] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.215025 23645 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:51.220927 23736 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.221524 23736 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.223680 23736 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.226259 23638 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.226873 23638 ts_tablet_manager.cc:1403] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.088s	user 0.006s	sys 0.005s
I20260709 13:06:51.230371 23638 raft_consensus.cc:359] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.231120 23638 raft_consensus.cc:385] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.231164 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.232000 23445 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:51.231421 23638 raft_consensus.cc:740] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.235095 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:51.235603 23638 consensus_queue.cc:260] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.235888 23333 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:51.237179 23645 tablet_bootstrap.cc:654] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.237360 23895 raft_consensus.cc:493] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.237809 23638 ts_tablet_manager.cc:1434] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 13:06:51.250610 23638 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.237875 23895 raft_consensus.cc:515] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.260118 23895 leader_election.cc:290] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:51.261276 23638 tablet_bootstrap.cc:654] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.262218 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:51.265786 23728 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.266393 23728 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.266979 23267 catalog_manager.cc:5697] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.269898 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.237062 23956 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.282656 23728 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.283985 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
W20260709 13:06:51.271831 23556 leader_election.cc:343] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:51.281777 23333 leader_election.cc:343] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.285975 23956 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.289149 23956 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.304992 23945 raft_consensus.cc:493] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.284775 23442 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
W20260709 13:06:51.293025 23558 leader_election.cc:343] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.294723 23881 raft_consensus.cc:493] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.305773 23945 raft_consensus.cc:515] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.313028 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:51.320794 23558 leader_election.cc:304] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.327255 23638 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.327814 23638 ts_tablet_manager.cc:1403] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:06:51.314049 23881 raft_consensus.cc:515] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.324523 23866 raft_consensus.cc:2749] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.322645 23517 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:51.332899 23945 leader_election.cc:290] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.335793 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.337337 23945 consensus_queue.cc:1048] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:51.337693 23558 leader_election.cc:343] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.341566 23639 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:51.342118 23639 ts_tablet_manager.cc:1403] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.176s	user 0.025s	sys 0.031s
I20260709 13:06:51.342566 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.343508 23442 raft_consensus.cc:2468] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:51.346711 23639 raft_consensus.cc:359] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.347482 23639 raft_consensus.cc:385] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.347788 23639 raft_consensus.cc:740] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.348124 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.348635 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.349125 23592 raft_consensus.cc:2468] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:51.348588 23639 consensus_queue.cc:260] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.351830 23639 ts_tablet_manager.cc:1434] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:06:51.330744 23881 leader_election.cc:290] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:06:51.352864 23410 leader_election.cc:343] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.353292 23556 leader_election.cc:304] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.355010 23866 raft_consensus.cc:2804] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.355651 23866 raft_consensus.cc:493] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.356151 23866 raft_consensus.cc:3060] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.354496 23885 consensus_queue.cc:1048] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:51.374478 23408 leader_election.cc:304] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.374701 23945 consensus_queue.cc:1048] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.383901 23881 raft_consensus.cc:2804] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.385043 23881 raft_consensus.cc:493] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.385499 23881 raft_consensus.cc:3060] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.402734 23645 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.403314 23645 ts_tablet_manager.cc:1403] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.189s	user 0.023s	sys 0.053s
W20260709 13:06:51.404538 23482 consensus_peers.cc:597] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 -> Peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Couldn't send request to peer ceb56da8fd5145648a898f5487219c6e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.376610 23639 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:51.379539 23638 raft_consensus.cc:359] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.407047 23638 raft_consensus.cc:385] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.407356 23638 raft_consensus.cc:740] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.407970 23638 consensus_queue.cc:260] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.409610 23638 ts_tablet_manager.cc:1434] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
I20260709 13:06:51.410555 23638 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.413340 23645 raft_consensus.cc:359] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.414100 23956 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.414090 23645 raft_consensus.cc:385] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.417688 23645 raft_consensus.cc:740] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.418418 23645 consensus_queue.cc:260] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.420481 23645 ts_tablet_manager.cc:1434] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:06:51.423575 23638 tablet_bootstrap.cc:654] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.424952 23639 tablet_bootstrap.cc:654] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.425928 23645 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
W20260709 13:06:51.429906 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 114984us
I20260709 13:06:51.436491 23945 consensus_queue.cc:1048] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.442515 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:51.443183 23866 raft_consensus.cc:515] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.449057 23333 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
W20260709 13:06:51.452167 23333 leader_election.cc:343] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.416280 23956 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.453835 23956 raft_consensus.cc:2804] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.454176 23736 raft_consensus.cc:493] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.463477 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:51.458004 23736 raft_consensus.cc:515] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.466663 23736 leader_election.cc:290] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.467643 23368 raft_consensus.cc:3060] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.469537 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:51.478749 23443 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.489421 23645 tablet_bootstrap.cc:654] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.494563 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:51.504685 23866 leader_election.cc:290] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.507627 23443 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:51.454200 23956 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.524273 23638 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.524890 23638 ts_tablet_manager.cc:1403] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.115s	user 0.008s	sys 0.005s
I20260709 13:06:51.527456 23881 raft_consensus.cc:515] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.527758 23638 raft_consensus.cc:359] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.528400 23638 raft_consensus.cc:385] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.528728 23442 raft_consensus.cc:2393] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ceb56da8fd5145648a898f5487219c6e in current term 1: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:51.528689 23638 raft_consensus.cc:740] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.529505 23638 consensus_queue.cc:260] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.531940 23638 ts_tablet_manager.cc:1434] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:06:51.532907 23638 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.537612 23368 raft_consensus.cc:2468] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:51.539067 23638 tablet_bootstrap.cc:654] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.519469 23956 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.553962 23333 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:51.557765 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.558673 23592 raft_consensus.cc:2393] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:51.566042 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.566967 23444 raft_consensus.cc:2393] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:51.561749 23736 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.567687 23736 raft_consensus.cc:697] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:51.568571 23736 consensus_queue.cc:237] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.569793 23639 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:51.570459 23639 ts_tablet_manager.cc:1403] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.194s	user 0.012s	sys 0.012s
I20260709 13:06:51.565956 23642 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:51.571287 23642 ts_tablet_manager.cc:1403] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.372s	user 0.024s	sys 0.041s
I20260709 13:06:51.575874 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:51.576730 23368 raft_consensus.cc:2393] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:51.578398 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ce58a33542c395d1e4e41fe71ea7" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:51.578608 23639 raft_consensus.cc:359] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.579298 23639 raft_consensus.cc:385] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.579303 23592 raft_consensus.cc:2393] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:51.581430 23333 leader_election.cc:304] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:51.579617 23639 raft_consensus.cc:740] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.582477 23970 raft_consensus.cc:2749] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.583046 23639 consensus_queue.cc:260] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.576653 23558 leader_election.cc:304] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:51.584656 23408 leader_election.cc:304] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.585680 23865 raft_consensus.cc:2749] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.586484 23639 ts_tablet_manager.cc:1434] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:06:51.589076 23642 raft_consensus.cc:359] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.590551 23642 raft_consensus.cc:385] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.590826 23642 raft_consensus.cc:740] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.591426 23642 consensus_queue.cc:260] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.592408 23639 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:51.574012 23881 leader_election.cc:290] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:06:51.592324 23556 consensus_peers.cc:597] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.605129 23645 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.605749 23645 ts_tablet_manager.cc:1403] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.180s	user 0.018s	sys 0.011s
I20260709 13:06:51.605803 23896 raft_consensus.cc:493] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.606379 23896 raft_consensus.cc:515] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.608400 23896 leader_election.cc:290] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.592478 23866 raft_consensus.cc:2804] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.610205 23866 raft_consensus.cc:697] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:51.608484 23645 raft_consensus.cc:359] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.611402 23645 raft_consensus.cc:385] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.611824 23645 raft_consensus.cc:740] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.612758 23645 consensus_queue.cc:260] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.618563 23642 ts_tablet_manager.cc:1434] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.002s
I20260709 13:06:51.634809 23645 ts_tablet_manager.cc:1434] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:06:51.636656 23642 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:51.642050 23866 consensus_queue.cc:237] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.633785 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.645963 23445 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:51.649204 23645 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:51.657812 23638 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.658437 23638 ts_tablet_manager.cc:1403] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.016s
I20260709 13:06:51.658779 23333 leader_election.cc:304] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:51.652997 23945 raft_consensus.cc:493] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.662395 23964 raft_consensus.cc:2804] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.675107 23265 catalog_manager.cc:5697] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.674336 23964 raft_consensus.cc:493] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.683343 23964 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.671679 23945 raft_consensus.cc:515] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.686193 23956 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.688011 23945 leader_election.cc:290] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.688287 23956 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:51.698284 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.700021 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:51.702021 23556 leader_election.cc:343] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:51.702720 23556 leader_election.cc:343] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.703159 23556 leader_election.cc:304] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:51.703671 23370 raft_consensus.cc:3060] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.706326 23639 tablet_bootstrap.cc:654] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.672538 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.707608 23592 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
I20260709 13:06:51.709028 23913 raft_consensus.cc:2749] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.712970 23638 raft_consensus.cc:359] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.713743 23638 raft_consensus.cc:385] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.714001 23638 raft_consensus.cc:740] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.714658 23638 consensus_queue.cc:260] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.716289 23638 ts_tablet_manager.cc:1434] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.003s
I20260709 13:06:51.733027 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:51.733671 23443 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.738334 23645 tablet_bootstrap.cc:654] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.737457 23266 catalog_manager.cc:5697] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.738965 23638 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.749878 23964 raft_consensus.cc:515] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.760865 23831 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.761499 23831 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.766937 23881 consensus_queue.cc:1048] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.764010 23831 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:51.772503 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.774097 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:51.774303 23482 leader_election.cc:343] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.774780 23519 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.779292 23642 tablet_bootstrap.cc:654] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.781569 23443 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:51.784662 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:51.785686 23445 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:51.787324 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:51.789588 23333 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:51.790555 23728 raft_consensus.cc:2804] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.791342 23728 raft_consensus.cc:697] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:51.792589 23728 consensus_queue.cc:237] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.810482 23638 tablet_bootstrap.cc:654] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.782986 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:51.782258 23964 leader_election.cc:290] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.811590 23519 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:51.813807 23588 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.822589 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.808678 23445 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:51.827410 23333 leader_election.cc:304] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:51.829751 23728 raft_consensus.cc:2804] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:51.829913 23484 leader_election.cc:343] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.830605 23484 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.830345 23728 raft_consensus.cc:697] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:51.833444 23728 consensus_queue.cc:237] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.833961 23831 raft_consensus.cc:2749] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.836252 23881 consensus_queue.cc:1048] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.842339 23639 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:51.843250 23639 ts_tablet_manager.cc:1403] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.256s	user 0.007s	sys 0.013s
I20260709 13:06:51.846678 23639 raft_consensus.cc:359] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.849779 23639 raft_consensus.cc:385] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.850013 23639 raft_consensus.cc:740] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.852627 23588 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
I20260709 13:06:51.852816 23639 consensus_queue.cc:260] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.858006 23639 ts_tablet_manager.cc:1434] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:51.862030 23831 raft_consensus.cc:493] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.864948 23831 raft_consensus.cc:515] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.865579 23639 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:51.867036 23831 leader_election.cc:290] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:51.864261 23645 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.876040 23645 ts_tablet_manager.cc:1403] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.227s	user 0.026s	sys 0.020s
I20260709 13:06:51.885771 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:51.888880 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "376a2904d9ff484195a63051a29b8a0d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.894218 23645 raft_consensus.cc:359] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.894942 23645 raft_consensus.cc:385] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.895212 23645 raft_consensus.cc:740] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.895984 23645 consensus_queue.cc:260] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:51.897471 23639 tablet_bootstrap.cc:654] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.905011 23265 catalog_manager.cc:5697] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.910642 23645 ts_tablet_manager.cc:1434] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.012s
I20260709 13:06:51.912443 23831 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.913609 23265 catalog_manager.cc:5697] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.913616 23639 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:51.917448 23639 ts_tablet_manager.cc:1403] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.002s
I20260709 13:06:51.914302 23831 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.919914 23639 raft_consensus.cc:359] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.920465 23639 raft_consensus.cc:385] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.920698 23831 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:51.914739 23484 leader_election.cc:304] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.924427 23831 raft_consensus.cc:2749] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.914453 23645 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:51.930907 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:51.931810 23443 raft_consensus.cc:2393] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:51.937580 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:51.920703 23639 raft_consensus.cc:740] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.939054 23639 consensus_queue.cc:260] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:51.940404 23484 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:51.941002 23638 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:51.941488 23638 ts_tablet_manager.cc:1403] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.218s	user 0.020s	sys 0.008s
I20260709 13:06:51.940658 23645 tablet_bootstrap.cc:654] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.942018 23831 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.942301 23639 ts_tablet_manager.cc:1434] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.001s
I20260709 13:06:51.943971 23638 raft_consensus.cc:359] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.944558 23638 raft_consensus.cc:385] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.944788 23638 raft_consensus.cc:740] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.946033 23639 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:51.947750 23642 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:51.951074 23642 ts_tablet_manager.cc:1403] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.315s	user 0.018s	sys 0.011s
I20260709 13:06:51.945626 23638 consensus_queue.cc:260] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:51.954981 23638 ts_tablet_manager.cc:1434] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:51.956315 23638 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:51.963893 23831 raft_consensus.cc:2749] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.965322 23642 raft_consensus.cc:359] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.967875 23639 tablet_bootstrap.cc:654] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.968520 23645 tablet_bootstrap.cc:492] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:51.975884 23638 tablet_bootstrap.cc:654] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.977082 23645 ts_tablet_manager.cc:1403] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.009s
I20260709 13:06:51.976485 23642 raft_consensus.cc:385] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.979683 23642 raft_consensus.cc:740] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.980489 23642 consensus_queue.cc:260] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.980369 23645 raft_consensus.cc:359] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.982365 23645 raft_consensus.cc:385] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.982690 23645 raft_consensus.cc:740] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.983837 23642 ts_tablet_manager.cc:1434] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:06:51.985241 23642 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:51.983742 23645 consensus_queue.cc:260] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:51.995005 23645 ts_tablet_manager.cc:1434] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:06:51.997843 23642 tablet_bootstrap.cc:654] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.998028 23519 raft_consensus.cc:3060] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.999779 23639 tablet_bootstrap.cc:492] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:52.000344 23639 ts_tablet_manager.cc:1403] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.055s	user 0.005s	sys 0.010s
I20260709 13:06:52.002673 23638 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:52.003203 23638 ts_tablet_manager.cc:1403] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.047s	user 0.005s	sys 0.007s
I20260709 13:06:52.003669 23639 raft_consensus.cc:359] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.004385 23639 raft_consensus.cc:385] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.004658 23639 raft_consensus.cc:740] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.005570 23639 consensus_queue.cc:260] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.006083 23638 raft_consensus.cc:359] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.006883 23638 raft_consensus.cc:385] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.007145 23638 raft_consensus.cc:740] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.007982 23639 ts_tablet_manager.cc:1434] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:06:52.007939 23638 consensus_queue.cc:260] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.009073 23639 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:52.017776 23638 ts_tablet_manager.cc:1434] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:06:52.018850 23638 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:52.023360 23728 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.023940 23728 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:52.026075 23728 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:52.026860 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:52.027751 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:52.031080 23333 leader_election.cc:343] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.032167 23333 leader_election.cc:343] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.032621 23333 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:52.034051 23964 raft_consensus.cc:2749] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.035321 23638 tablet_bootstrap.cc:654] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.036438 23639 tablet_bootstrap.cc:654] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.053714 23645 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:52.060803 23831 raft_consensus.cc:493] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.061862 23831 raft_consensus.cc:515] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.137506 23913 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.138226 23913 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.140633 23913 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:52.145125 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:52.145756 23368 raft_consensus.cc:2468] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:52.147017 23558 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:52.147830 23964 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.148231 23913 raft_consensus.cc:2804] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.148348 23964 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.148914 23913 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.149435 23913 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.150842 23964 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:52.153630 23831 leader_election.cc:290] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:52.154531 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.155238 23588 raft_consensus.cc:2468] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:52.156683 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:52.157414 23368 raft_consensus.cc:2468] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:52.157572 23482 leader_election.cc:304] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:52.158459 23979 raft_consensus.cc:2804] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.158854 23979 raft_consensus.cc:493] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.159157 23979 raft_consensus.cc:3060] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.159973 23913 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.165269 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:52.165679 23642 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:52.166309 23642 ts_tablet_manager.cc:1403] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.181s	user 0.019s	sys 0.033s
I20260709 13:06:52.167484 23638 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:52.168007 23638 ts_tablet_manager.cc:1403] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.149s	user 0.011s	sys 0.006s
I20260709 13:06:52.169531 23642 raft_consensus.cc:359] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.170464 23642 raft_consensus.cc:385] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.170938 23642 raft_consensus.cc:740] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.170466 23638 raft_consensus.cc:359] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.171705 23638 raft_consensus.cc:385] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.172091 23638 raft_consensus.cc:740] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.171808 23642 consensus_queue.cc:260] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.172978 23638 consensus_queue.cc:260] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.174537 23642 ts_tablet_manager.cc:1434] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:52.175248 23638 ts_tablet_manager.cc:1434] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:52.177160 23638 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:52.178351 23642 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
W20260709 13:06:52.169972 23333 leader_election.cc:343] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.180231 23645 tablet_bootstrap.cc:654] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.182544 23979 raft_consensus.cc:515] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.185248 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:52.185531 23979 leader_election.cc:290] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:52.185350 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:52.187994 23588 raft_consensus.cc:3060] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.189286 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c5c2c534bbd4c909a5d8fd8e8ce52e5" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:52.189888 23368 raft_consensus.cc:3060] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.196894 23642 tablet_bootstrap.cc:654] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.236649 23882 raft_consensus.cc:493] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.237438 23882 raft_consensus.cc:515] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.219969 23728 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:06:52.239581 23964 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.242615 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:52.243400 23367 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.244161 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:52.242631 23882 leader_election.cc:290] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
W20260709 13:06:52.253274 23408 leader_election.cc:343] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.253118 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.254098 23592 raft_consensus.cc:2468] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:52.254556 23368 raft_consensus.cc:2468] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:52.257052 23408 leader_election.cc:304] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:52.264154 23639 tablet_bootstrap.cc:492] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:52.265707 23639 ts_tablet_manager.cc:1403] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.257s	user 0.025s	sys 0.007s
I20260709 13:06:52.268184 23882 raft_consensus.cc:2804] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.268813 23882 raft_consensus.cc:493] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.269202 23882 raft_consensus.cc:3060] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.261130 23588 raft_consensus.cc:2468] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:52.273038 23997 consensus_queue.cc:1048] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.274514 23482 leader_election.cc:304] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:52.276804 23821 raft_consensus.cc:2804] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.277593 23821 raft_consensus.cc:697] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:52.278848 23821 consensus_queue.cc:237] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.276661 23638 tablet_bootstrap.cc:654] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.271302 23639 raft_consensus.cc:359] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.283404 23639 raft_consensus.cc:385] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.283886 23639 raft_consensus.cc:740] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.284947 23639 consensus_queue.cc:260] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.294893 23645 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:52.295441 23645 ts_tablet_manager.cc:1403] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.256s	user 0.024s	sys 0.015s
I20260709 13:06:52.291963 23913 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:52.291934 23367 raft_consensus.cc:2468] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
W20260709 13:06:52.322010 23556 leader_election.cc:343] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.324298 23997 raft_consensus.cc:493] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.325409 23587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.325779 23997 raft_consensus.cc:515] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
W20260709 13:06:52.328248 23333 leader_election.cc:343] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.326333 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:52.330101 23333 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:52.330911 23978 consensus_queue.cc:1048] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:52.338003 23556 leader_election.cc:343] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.345471 23639 ts_tablet_manager.cc:1434] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.079s	user 0.021s	sys 0.015s
I20260709 13:06:52.346742 23639 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:52.347997 23997 leader_election.cc:290] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:52.317937 23728 consensus_queue.cc:1048] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:06:52.353005 23558 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:52.349161 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:52.339490 23978 raft_consensus.cc:2749] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.360481 23997 raft_consensus.cc:2804] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.373674 23642 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:52.374444 23997 raft_consensus.cc:697] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:52.374291 23642 ts_tablet_manager.cc:1403] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.196s	user 0.018s	sys 0.033s
I20260709 13:06:52.374249 23945 consensus_queue.cc:1048] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.375451 23997 consensus_queue.cc:237] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.398425 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f7c7f831d14ec5870cb93e9b1f9f02" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:52.401044 23642 raft_consensus.cc:359] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.401944 23642 raft_consensus.cc:385] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.402251 23642 raft_consensus.cc:740] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.403038 23642 consensus_queue.cc:260] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.403950 23265 catalog_manager.cc:5697] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.416641 23645 raft_consensus.cc:359] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.418325 23645 raft_consensus.cc:385] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.418627 23645 raft_consensus.cc:740] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.419562 23588 raft_consensus.cc:3060] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.432518 23881 raft_consensus.cc:493] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.433161 23881 raft_consensus.cc:515] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.436187 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:52.436933 23519 raft_consensus.cc:2468] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:52.437270 23645 consensus_queue.cc:260] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.438445 23408 leader_election.cc:304] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:52.439005 23645 ts_tablet_manager.cc:1434] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.143s	user 0.006s	sys 0.001s
I20260709 13:06:52.439765 24007 raft_consensus.cc:2804] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.440210 24007 raft_consensus.cc:493] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.440583 24007 raft_consensus.cc:3060] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.444141 23645 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:52.444902 23881 leader_election.cc:290] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:52.445667 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.446429 23593 raft_consensus.cc:2468] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:52.436662 23882 raft_consensus.cc:515] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.455387 23882 leader_election.cc:290] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:52.457127 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:52.457391 23638 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:52.457422 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4743addeb1314388b9469d11e0ff9385" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:52.457830 23519 raft_consensus.cc:3060] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.458284 23638 ts_tablet_manager.cc:1403] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.281s	user 0.017s	sys 0.012s
I20260709 13:06:52.459663 24007 raft_consensus.cc:515] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.462940 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:52.463624 23517 raft_consensus.cc:3060] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.464507 23642 ts_tablet_manager.cc:1434] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 13:06:52.465320 23593 raft_consensus.cc:3060] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.465826 23642 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:52.467303 23519 raft_consensus.cc:2468] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:52.469188 23408 leader_election.cc:304] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:52.472271 23882 raft_consensus.cc:2804] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.474432 23639 tablet_bootstrap.cc:654] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.473263 23882 raft_consensus.cc:697] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:52.475585 23882 consensus_queue.cc:237] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.477399 23517 raft_consensus.cc:2468] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:52.480352 23408 leader_election.cc:304] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:52.481380 23881 raft_consensus.cc:2804] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.482196 23881 raft_consensus.cc:697] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:52.483161 23881 consensus_queue.cc:237] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.490701 23645 tablet_bootstrap.cc:654] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.496809 23593 raft_consensus.cc:2468] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
W20260709 13:06:52.499046 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 139844us
W20260709 13:06:52.506896 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 146303us
I20260709 13:06:52.463652 23638 raft_consensus.cc:359] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.508508 23638 raft_consensus.cc:385] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.509691 23638 raft_consensus.cc:740] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.513293 23638 consensus_queue.cc:260] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.514485 23556 leader_election.cc:304] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:52.525906 23268 catalog_manager.cc:5697] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:52.559213 23410 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 62799us
W20260709 13:06:52.566252 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 56764.8us
I20260709 13:06:52.535096 24007 leader_election.cc:290] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:52.569403 23593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8c839e30fa42ee9ff04b9ba759fc85" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:52.569648 23997 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.594885 23593 raft_consensus.cc:3060] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.596340 23997 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:52.622623 23997 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:52.627530 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:52.628253 23519 raft_consensus.cc:2468] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:52.629590 23556 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:52.631677 24020 raft_consensus.cc:2804] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.633085 24020 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.642206 24007 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.647571 23645 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:52.651675 23642 tablet_bootstrap.cc:654] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.651975 23645 ts_tablet_manager.cc:1403] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.208s	user 0.015s	sys 0.006s
I20260709 13:06:52.654795 23645 raft_consensus.cc:359] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.655463 23645 raft_consensus.cc:385] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.655746 23645 raft_consensus.cc:740] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.647645 24007 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
W20260709 13:06:52.658438 24017 log.cc:927] Time spent T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:06:52.659931 23587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.662696 23639 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:52.663120 24007 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:52.663255 23639 ts_tablet_manager.cc:1403] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.317s	user 0.015s	sys 0.002s
I20260709 13:06:52.637948 24020 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.663663 23266 catalog_manager.cc:5697] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.666927 23639 raft_consensus.cc:359] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:52.670740 23639 raft_consensus.cc:385] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.671073 23639 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.672042 23639 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:52.672503 23645 consensus_queue.cc:260] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.674870 23638 ts_tablet_manager.cc:1434] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.216s	user 0.017s	sys 0.001s
I20260709 13:06:52.675390 23866 raft_consensus.cc:493] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.675925 23866 raft_consensus.cc:515] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:52.676471 23638 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:52.669406 23266 catalog_manager.cc:5697] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.677666 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:52.682647 23645 ts_tablet_manager.cc:1434] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:06:52.683624 23645 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
W20260709 13:06:52.686650 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 119172us
I20260709 13:06:52.690979 23639 ts_tablet_manager.cc:1434] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:06:52.707638 23866 leader_election.cc:290] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:52.704697 24008 consensus_queue.cc:1048] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:06:52.711508 24028 consensus_queue.cc:1048] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.694311 23593 raft_consensus.cc:2468] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:52.727988 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:52.731016 23368 raft_consensus.cc:2393] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:52.703439 24022 raft_consensus.cc:493] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.743072 24022 raft_consensus.cc:515] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:52.745214 24022 leader_election.cc:290] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:52.753957 23639 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:52.693912 23979 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:52.718974 23558 leader_election.cc:343] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.846789 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:52.846985 23587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:52.850368 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:52.854658 24009 raft_consensus.cc:2749] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.855676 23645 tablet_bootstrap.cc:654] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.716804 24020 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
W20260709 13:06:52.859999 23333 leader_election.cc:343] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.878468 23333 leader_election.cc:343] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.878988 23333 leader_election.cc:304] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:52.776541 23979 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:52.863912 23558 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 144965us
I20260709 13:06:52.901796 23410 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:52.911422 23882 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.912346 23638 tablet_bootstrap.cc:654] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.890793 23639 tablet_bootstrap.cc:654] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.897840 23642 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:52.899401 24020 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:52.886621 24022 raft_consensus.cc:2749] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.889209 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:52.903805 23556 leader_election.cc:343] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.905581 23985 log.cc:927] Time spent T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:52.907202 23979 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:52.907050 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:52.916043 23642 ts_tablet_manager.cc:1403] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.451s	user 0.029s	sys 0.010s
I20260709 13:06:52.945529 23517 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:52.972129 23556 leader_election.cc:343] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.973197 23556 leader_election.cc:304] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:52.976311 23866 raft_consensus.cc:2749] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.005505 23638 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:53.005303 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.006011 23638 ts_tablet_manager.cc:1403] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.330s	user 0.022s	sys 0.023s
I20260709 13:06:53.007488 24012 raft_consensus.cc:493] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.008056 24012 raft_consensus.cc:515] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.010133 24012 leader_election.cc:290] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:53.010421 23645 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:53.011008 23645 ts_tablet_manager.cc:1403] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.328s	user 0.020s	sys 0.016s
I20260709 13:06:53.010845 23638 raft_consensus.cc:359] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.011592 23638 raft_consensus.cc:385] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.011885 23638 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.013358 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.013799 23645 raft_consensus.cc:359] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.014518 23645 raft_consensus.cc:385] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.014777 23645 raft_consensus.cc:740] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.015404 23645 consensus_queue.cc:260] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.015981 23638 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.017463 23638 ts_tablet_manager.cc:1434] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:53.017833 23645 ts_tablet_manager.cc:1434] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:06:53.018378 23638 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:53.018736 23645 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:53.024127 23638 tablet_bootstrap.cc:654] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.024574 23645 tablet_bootstrap.cc:654] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.038676 23482 leader_election.cc:343] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.046988 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:53.036378 23558 leader_election.cc:343] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.065382 23642 raft_consensus.cc:359] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.066445 23642 raft_consensus.cc:385] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.066792 23642 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.067910 23591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.067677 23642 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.069276 24012 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.069965 24012 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.081063 24026 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.081662 24026 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.081773 24036 consensus_queue.cc:1048] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.072556 24012 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:53.091959 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:53.093798 23333 leader_election.cc:343] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.095814 24009 consensus_queue.cc:1048] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.107945 23642 ts_tablet_manager.cc:1434] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.176s	user 0.008s	sys 0.000s
I20260709 13:06:53.108948 23642 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:53.111574 23370 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:53.137377 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:53.141932 23556 leader_election.cc:343] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.149786 24016 consensus_queue.cc:1048] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:06:53.157099 23484 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:53.157956 24036 raft_consensus.cc:2804] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.158325 24036 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.158638 24036 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.162177 24026 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:53.143751 23882 raft_consensus.cc:2749] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.169034 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:53.169034 23517 raft_consensus.cc:2468] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:53.167125 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:53.170298 23366 raft_consensus.cc:2468] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:53.171629 23556 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:53.174925 23645 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:53.175448 23645 ts_tablet_manager.cc:1403] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.157s	user 0.021s	sys 0.014s
I20260709 13:06:53.143980 24044 consensus_queue.cc:1048] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:06:53.175926 23865 consensus_queue.cc:1048] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.190304 23558 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:53.191241 24026 raft_consensus.cc:2804] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.191623 24026 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.191931 24026 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.193230 24037 consensus_queue.cc:1048] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.199707 24026 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.190219 23645 raft_consensus.cc:359] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.201557 23645 raft_consensus.cc:385] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.201817 23645 raft_consensus.cc:740] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.202519 24020 raft_consensus.cc:2804] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.202450 23645 consensus_queue.cc:260] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.203095 24020 raft_consensus.cc:697] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:53.209769 23645 ts_tablet_manager.cc:1434] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 13:06:53.204319 24026 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:53.227075 24026 consensus_queue.cc:1048] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.230605 23639 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:53.231196 23639 ts_tablet_manager.cc:1403] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.478s	user 0.036s	sys 0.018s
I20260709 13:06:53.234791 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:53.247529 23556 leader_election.cc:343] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.247318 24036 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.249897 23516 raft_consensus.cc:2393] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:53.244451 23639 raft_consensus.cc:359] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.251855 23639 raft_consensus.cc:385] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.252311 23639 raft_consensus.cc:740] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.253350 23639 consensus_queue.cc:260] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.253815 23442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:53.203981 24020 consensus_queue.cc:237] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.256582 23638 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:53.257217 23638 ts_tablet_manager.cc:1403] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.239s	user 0.018s	sys 0.005s
I20260709 13:06:53.259470 23442 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.260241 23638 raft_consensus.cc:359] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.261018 23638 raft_consensus.cc:385] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.261270 23638 raft_consensus.cc:740] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.261904 23638 consensus_queue.cc:260] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.264042 23638 ts_tablet_manager.cc:1434] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:06:53.265033 23638 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:53.266342 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.266942 23366 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.273999 23366 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:53.280010 23638 tablet_bootstrap.cc:654] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.280939 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.281741 23366 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.283327 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.255487 24036 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:53.328429 23645 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:53.328507 23639 ts_tablet_manager.cc:1434] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
I20260709 13:06:53.272552 23442 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:53.330116 23639 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:53.335098 23642 tablet_bootstrap.cc:654] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.339432 23438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.324357 24044 consensus_queue.cc:1048] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:06:53.336958 23865 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.349336 23865 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.350801 23438 raft_consensus.cc:2393] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:53.354907 23484 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:53.356576 24037 raft_consensus.cc:2804] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.385192 23645 tablet_bootstrap.cc:654] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.371454 24037 raft_consensus.cc:697] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:53.393880 24019 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.397114 23366 raft_consensus.cc:2468] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:53.398255 23865 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:53.404278 23638 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:53.404798 23638 ts_tablet_manager.cc:1403] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.140s	user 0.018s	sys 0.035s
I20260709 13:06:53.401321 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:53.417263 23639 tablet_bootstrap.cc:654] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.394536 24019 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.402240 23558 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
W20260709 13:06:53.440220 23333 leader_election.cc:343] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.449170 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:53.454962 23558 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 52819us
I20260709 13:06:53.457208 24026 raft_consensus.cc:2804] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.457926 24026 raft_consensus.cc:697] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:53.459025 24026 consensus_queue.cc:237] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.461011 24009 raft_consensus.cc:493] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.461714 24009 raft_consensus.cc:515] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.464486 24009 leader_election.cc:290] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:53.443240 23333 leader_election.cc:304] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:53.469846 23728 raft_consensus.cc:2749] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.472056 24019 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
W20260709 13:06:53.474728 24053 log.cc:927] Time spent T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:06:53.441972 24037 consensus_queue.cc:237] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.468672 23438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.419387 23367 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:53.485980 23410 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:53.490731 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:53.492827 23333 leader_election.cc:343] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.459431 23268 catalog_manager.cc:5697] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.501302 23865 raft_consensus.cc:2804] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:53.501936 23865 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.502370 23865 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:53.512063 23556 leader_election.cc:343] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.514321 23333 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:53.516201 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:53.516916 23367 raft_consensus.cc:2468] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:53.521490 23865 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.522895 23728 raft_consensus.cc:2749] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.526284 23558 leader_election.cc:304] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:53.529803 24009 raft_consensus.cc:2804] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.530236 24009 raft_consensus.cc:493] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.530668 24009 raft_consensus.cc:3060] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.533605 23865 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:53.534746 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:53.535492 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.536104 23367 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.543256 23367 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:53.544711 23410 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:53.545154 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:53.545657 23865 raft_consensus.cc:2804] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.546584 24009 raft_consensus.cc:515] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.551491 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.552170 23366 raft_consensus.cc:3060] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.559897 23366 raft_consensus.cc:2468] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:53.548753 23865 raft_consensus.cc:697] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:53.563460 23438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:53.563153 23865 consensus_queue.cc:237] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:53.565040 23556 leader_election.cc:343] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.561949 24009 leader_election.cc:290] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:53.570932 23642 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:53.571462 23642 ts_tablet_manager.cc:1403] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.463s	user 0.017s	sys 0.000s
I20260709 13:06:53.574741 23558 leader_election.cc:304] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:53.575729 24009 raft_consensus.cc:2804] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.576365 24009 raft_consensus.cc:697] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:53.578444 24009 consensus_queue.cc:237] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.586961 23645 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:53.587598 23645 ts_tablet_manager.cc:1403] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.259s	user 0.040s	sys 0.070s
I20260709 13:06:53.591728 23645 raft_consensus.cc:359] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.593187 23645 raft_consensus.cc:385] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.593890 23645 raft_consensus.cc:740] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.594901 23645 consensus_queue.cc:260] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.597705 23645 ts_tablet_manager.cc:1434] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:53.602418 23645 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:53.608008 23267 catalog_manager.cc:5697] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 2, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.611963 23639 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:53.612509 23639 ts_tablet_manager.cc:1403] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.283s	user 0.018s	sys 0.034s
I20260709 13:06:53.550853 23638 raft_consensus.cc:359] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.613866 23638 raft_consensus.cc:385] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.614283 23638 raft_consensus.cc:740] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.615238 23642 raft_consensus.cc:359] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.615104 23638 consensus_queue.cc:260] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.615811 23642 raft_consensus.cc:385] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.615265 23645 tablet_bootstrap.cc:654] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.616140 23642 raft_consensus.cc:740] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.619182 23638 ts_tablet_manager.cc:1434] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.214s	user 0.008s	sys 0.001s
I20260709 13:06:53.620265 23638 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:53.617107 23642 consensus_queue.cc:260] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.625011 23642 ts_tablet_manager.cc:1434] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:06:53.625912 23642 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:53.630734 23638 tablet_bootstrap.cc:654] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.635733 23333 leader_election.cc:343] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.636222 23333 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:53.636510 23865 raft_consensus.cc:493] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.637185 23865 raft_consensus.cc:515] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.638602 23639 raft_consensus.cc:359] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.639380 23865 leader_election.cc:290] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:53.639935 23639 raft_consensus.cc:385] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.641621 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.641630 23367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf62d6f4e045f8b06e7170496fda73" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:53.642805 23367 raft_consensus.cc:2393] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:53.645205 23408 leader_election.cc:304] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:53.646250 23865 raft_consensus.cc:3060] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.646871 23638 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:53.647365 23638 ts_tablet_manager.cc:1403] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 13:06:53.647192 24029 raft_consensus.cc:2749] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.651587 23638 raft_consensus.cc:359] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.652251 23638 raft_consensus.cc:385] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.653505 23638 raft_consensus.cc:740] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.656634 23638 consensus_queue.cc:260] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.656708 24037 raft_consensus.cc:493] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.661460 23638 ts_tablet_manager.cc:1434] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:06:53.662464 23638 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:53.664743 24037 raft_consensus.cc:515] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.671340 23638 tablet_bootstrap.cc:654] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.640269 23639 raft_consensus.cc:740] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.680150 23639 consensus_queue.cc:260] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.680979 24037 leader_election.cc:290] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:53.681717 23865 raft_consensus.cc:2749] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.683107 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.684255 23438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.683836 23645 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:53.685215 23590 raft_consensus.cc:2468] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:53.686178 23645 ts_tablet_manager.cc:1403] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.005s
I20260709 13:06:53.687455 23642 tablet_bootstrap.cc:654] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.688939 24023 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.689467 24023 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.691847 24023 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:53.692229 23645 raft_consensus.cc:359] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.693156 23645 raft_consensus.cc:385] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.693488 23645 raft_consensus.cc:740] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.694185 23645 consensus_queue.cc:260] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
W20260709 13:06:53.694645 23482 leader_election.cc:343] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.695217 23639 ts_tablet_manager.cc:1434] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.000s
I20260709 13:06:53.695575 23482 leader_election.cc:304] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:53.683543 23266 catalog_manager.cc:5697] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.696235 23639 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:53.697256 23831 raft_consensus.cc:2804] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.697672 23831 raft_consensus.cc:493] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.698201 23831 raft_consensus.cc:3060] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.697661 23266 catalog_manager.cc:5697] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.702515 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:53.703341 23519 raft_consensus.cc:2393] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:53.720340 23639 tablet_bootstrap.cc:654] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.722880 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.731290 23333 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:53.731747 24009 consensus_queue.cc:1048] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.732890 23728 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.744289 23639 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:53.744900 23639 ts_tablet_manager.cc:1403] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.007s
I20260709 13:06:53.747536 23645 ts_tablet_manager.cc:1434] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.001s
I20260709 13:06:53.748622 23645 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:53.747944 23831 raft_consensus.cc:515] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.754444 23642 tablet_bootstrap.cc:492] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:53.754971 23642 ts_tablet_manager.cc:1403] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.129s	user 0.037s	sys 0.024s
I20260709 13:06:53.755272 23831 leader_election.cc:290] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:53.751664 23728 raft_consensus.cc:2749] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.757735 23642 raft_consensus.cc:359] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.758467 23642 raft_consensus.cc:385] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.758302 23639 raft_consensus.cc:359] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.758792 23642 raft_consensus.cc:740] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.759045 23639 raft_consensus.cc:385] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.759418 23639 raft_consensus.cc:740] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.759646 23642 consensus_queue.cc:260] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.760165 23639 consensus_queue.cc:260] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.762668 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:53.765347 23590 raft_consensus.cc:3060] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.766171 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038aa8132294ba7832fad08934a848f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:53.766806 23439 raft_consensus.cc:3060] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.767076 23639 ts_tablet_manager.cc:1434] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:06:53.769835 23639 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:53.770123 23642 ts_tablet_manager.cc:1434] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:06:53.773981 23642 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:53.776304 23639 tablet_bootstrap.cc:654] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.779879 24026 consensus_queue.cc:1048] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.781446 23642 tablet_bootstrap.cc:654] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.799204 23638 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
I20260709 13:06:53.799703 23638 ts_tablet_manager.cc:1403] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.000s
I20260709 13:06:53.804595 23638 raft_consensus.cc:359] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.805408 23638 raft_consensus.cc:385] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.805718 23638 raft_consensus.cc:740] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.806432 23439 raft_consensus.cc:2468] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:53.765556 23645 tablet_bootstrap.cc:654] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.824601 23590 raft_consensus.cc:2468] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:53.806399 23638 consensus_queue.cc:260] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.827219 23638 ts_tablet_manager.cc:1434] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:06:53.830217 23642 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:53.832180 23642 ts_tablet_manager.cc:1403] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.000s
I20260709 13:06:53.832603 23645 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:53.833200 23645 ts_tablet_manager.cc:1403] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.004s
I20260709 13:06:53.833335 23638 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:53.835362 23642 raft_consensus.cc:359] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.836292 23642 raft_consensus.cc:385] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.840142 23638 tablet_bootstrap.cc:654] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.844336 23645 raft_consensus.cc:359] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.845192 23645 raft_consensus.cc:385] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.845489 23645 raft_consensus.cc:740] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.830647 23482 leader_election.cc:304] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:53.848162 23831 raft_consensus.cc:2804] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.848652 23831 raft_consensus.cc:697] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:53.836686 23642 raft_consensus.cc:740] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.850960 23639 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:53.851583 23639 ts_tablet_manager.cc:1403] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.014s
I20260709 13:06:53.849483 23831 consensus_queue.cc:237] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.850658 23642 consensus_queue.cc:260] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.854457 23639 raft_consensus.cc:359] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.855209 23639 raft_consensus.cc:385] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.855530 23639 raft_consensus.cc:740] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.856555 23645 consensus_queue.cc:260] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.856773 23642 ts_tablet_manager.cc:1434] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:06:53.863701 23645 ts_tablet_manager.cc:1434] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:06:53.865152 23639 consensus_queue.cc:260] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.865484 23642 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:53.870628 23639 ts_tablet_manager.cc:1434] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:06:53.864822 23645 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:53.873317 23639 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:53.880702 23642 tablet_bootstrap.cc:654] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.881006 23639 tablet_bootstrap.cc:654] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.882169 24009 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.884114 23728 raft_consensus.cc:493] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.889164 23728 raft_consensus.cc:515] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:53.891757 23728 leader_election.cc:290] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:53.888094 24009 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.895975 24009 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:53.896335 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:53.897607 23366 raft_consensus.cc:2468] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:53.899348 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.901445 23558 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:53.902510 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.903431 23639 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:53.903962 23639 ts_tablet_manager.cc:1403] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.007s
W20260709 13:06:53.904614 23333 leader_election.cc:343] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.905023 24009 raft_consensus.cc:2804] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.905519 24009 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.905876 24009 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.908867 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:53.910703 23556 leader_election.cc:343] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.912868 23333 leader_election.cc:343] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.913530 23333 leader_election.cc:304] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:53.915664 23728 raft_consensus.cc:2749] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.886849 23645 tablet_bootstrap.cc:654] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.914898 23639 raft_consensus.cc:359] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.921758 23639 raft_consensus.cc:385] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.922083 23639 raft_consensus.cc:740] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.925472 23639 consensus_queue.cc:260] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.945950 23639 ts_tablet_manager.cc:1434] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 13:06:53.946930 24009 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.949514 23642 tablet_bootstrap.cc:492] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:53.950613 23642 ts_tablet_manager.cc:1403] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.015s
I20260709 13:06:53.952049 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:53.952737 23366 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.953552 23642 raft_consensus.cc:359] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.954313 23642 raft_consensus.cc:385] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.954698 23642 raft_consensus.cc:740] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.955497 23642 consensus_queue.cc:260] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.957979 23642 ts_tablet_manager.cc:1434] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:53.958951 23642 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:53.962169 23639 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:53.963685 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:53.968344 23642 tablet_bootstrap.cc:654] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.978035 23556 leader_election.cc:343] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.980746 24037 raft_consensus.cc:493] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.981603 24037 raft_consensus.cc:515] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:53.986462 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d36e08707f4d8295154d67b1450b0f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:53.987259 23440 raft_consensus.cc:2468] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:53.990123 24026 consensus_queue.cc:1048] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.990679 23831 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.991235 23831 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:53.995782 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d36e08707f4d8295154d67b1450b0f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:53.996971 23589 raft_consensus.cc:2468] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:54.002859 24029 raft_consensus.cc:493] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.003422 23370 raft_consensus.cc:1240] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Rejecting Update request from peer 12589c3399b641fc80f39058a475f8a1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:54.003583 24029 raft_consensus.cc:515] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:53.991578 23645 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:54.005194 23645 ts_tablet_manager.cc:1403] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.000s
I20260709 13:06:54.007527 23482 leader_election.cc:304] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:53.949640 24009 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:53.984728 24037 leader_election.cc:290] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:54.012503 23440 raft_consensus.cc:1240] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Rejecting Update request from peer 12589c3399b641fc80f39058a475f8a1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:54.013600 24003 raft_consensus.cc:2804] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.021019 23821 consensus_queue.cc:1059] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:54.024299 23821 raft_consensus.cc:3055] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.024703 23821 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:54.025604 23821 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.026512 23821 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.026695 24029 leader_election.cc:290] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:54.022193 24003 raft_consensus.cc:493] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.030107 24003 raft_consensus.cc:3060] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.033519 23639 tablet_bootstrap.cc:654] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.038674 23831 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:54.002425 23366 raft_consensus.cc:2468] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:54.120625 23558 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:54.131071 24026 raft_consensus.cc:493] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.132301 24026 raft_consensus.cc:515] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.135831 24009 raft_consensus.cc:2804] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.140312 23517 raft_consensus.cc:1275] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:06:54.140892 23517 pending_rounds.cc:85] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Aborting all ops after (but not including) 0
I20260709 13:06:54.141366 23517 pending_rounds.cc:107] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:54.140770 23645 raft_consensus.cc:359] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.141885 23517 raft_consensus.cc:2889] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:54.141970 23639 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:54.143292 23639 ts_tablet_manager.cc:1403] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.183s	user 0.012s	sys 0.005s
I20260709 13:06:54.143132 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:54.142031 23645 raft_consensus.cc:385] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.145531 23645 raft_consensus.cc:740] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.146438 23645 consensus_queue.cc:260] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.146991 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:54.148883 24009 raft_consensus.cc:697] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:54.150655 24009 consensus_queue.cc:237] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.171213 23268 catalog_manager.cc:5697] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.153875 24044 consensus_queue.cc:1048] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:54.156514 23333 leader_election.cc:343] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.159406 23519 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.165043 24026 leader_election.cc:290] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:54.167500 23642 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:54.169518 23638 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: No bootstrap required, opened a new log
W20260709 13:06:54.191565 23408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 44316.4us
I20260709 13:06:54.231822 23642 ts_tablet_manager.cc:1403] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.273s	user 0.019s	sys 0.032s
I20260709 13:06:54.232366 23728 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.233390 23728 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.235073 23638 ts_tablet_manager.cc:1403] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.407s	user 0.079s	sys 0.079s
I20260709 13:06:54.237567 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:54.238822 23369 raft_consensus.cc:2393] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:54.236640 23728 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:54.247440 23645 ts_tablet_manager.cc:1434] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.242s	user 0.008s	sys 0.006s
I20260709 13:06:54.242131 23639 raft_consensus.cc:359] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.252933 23639 raft_consensus.cc:385] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.253671 23639 raft_consensus.cc:740] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.252539 24003 raft_consensus.cc:515] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.258728 23645 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
I20260709 13:06:54.264163 23638 raft_consensus.cc:359] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.265327 23638 raft_consensus.cc:385] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.265946 23638 raft_consensus.cc:740] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.262569 23639 consensus_queue.cc:260] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.255968 23642 raft_consensus.cc:359] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.275732 23642 raft_consensus.cc:385] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.276542 23642 raft_consensus.cc:740] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.280591 23639 ts_tablet_manager.cc:1434] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.137s	user 0.011s	sys 0.000s
I20260709 13:06:54.283565 23885 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.284322 23885 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.286643 23882 raft_consensus.cc:493] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.287444 23882 raft_consensus.cc:515] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
W20260709 13:06:54.296224 23558 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 176091us
I20260709 13:06:54.296415 24003 leader_election.cc:290] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:54.304219 23885 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:54.303088 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:54.305189 23438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:54.333153 23369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:54.303683 23882 leader_election.cc:290] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:54.333966 23369 raft_consensus.cc:2468] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:54.306032 23265 catalog_manager.cc:5697] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.336587 24092 consensus_queue.cc:1048] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.338447 23366 raft_consensus.cc:1275] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:54.301690 23638 consensus_queue.cc:260] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.318210 24026 consensus_queue.cc:1048] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:06:54.340855 23728 raft_consensus.cc:493] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.305984 23484 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:54.301280 23642 consensus_queue.cc:260] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.346501 23728 raft_consensus.cc:515] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.305743 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d36e08707f4d8295154d67b1450b0f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
W20260709 13:06:54.350397 23333 leader_election.cc:343] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.352519 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e66e5d5789d4dc3a56f76f34b603bad" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:54.356475 23639 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:54.361435 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e66e5d5789d4dc3a56f76f34b603bad" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:54.362344 23518 raft_consensus.cc:2468] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
I20260709 13:06:54.379840 23642 ts_tablet_manager.cc:1434] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.147s	user 0.008s	sys 0.002s
I20260709 13:06:54.381563 23642 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:54.387145 23645 tablet_bootstrap.cc:654] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.388170 23590 raft_consensus.cc:3060] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.390513 23881 raft_consensus.cc:493] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.391088 23881 raft_consensus.cc:515] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.386053 23408 leader_election.cc:304] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:54.401588 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:54.399595 24044 raft_consensus.cc:2804] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.407042 24044 raft_consensus.cc:493] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.407480 24044 raft_consensus.cc:3060] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.409471 23638 ts_tablet_manager.cc:1434] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.174s	user 0.011s	sys 0.000s
I20260709 13:06:54.411219 23642 tablet_bootstrap.cc:654] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.414012 23728 leader_election.cc:290] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:54.409245 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d36e08707f4d8295154d67b1450b0f" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:54.415023 23440 raft_consensus.cc:3060] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.416114 23881 leader_election.cc:290] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
W20260709 13:06:54.416183 23484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 110531us
I20260709 13:06:54.425763 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:54.440819 23556 leader_election.cc:343] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.444623 23590 raft_consensus.cc:2468] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:54.447151 23519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:54.451682 23519 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 0.
W20260709 13:06:54.459774 23482 leader_election.cc:343] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.460189 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:54.465488 24044 raft_consensus.cc:515] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.490648 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:54.495817 23645 tablet_bootstrap.cc:492] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:54.496606 23645 ts_tablet_manager.cc:1403] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.238s	user 0.009s	sys 0.011s
W20260709 13:06:54.498376 23333 leader_election.cc:343] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.505774 24003 raft_consensus.cc:2749] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.515998 23821 consensus_queue.cc:1048] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.086s
I20260709 13:06:54.518735 23642 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:54.519228 23642 ts_tablet_manager.cc:1403] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.138s	user 0.016s	sys 0.025s
W20260709 13:06:54.513669 23408 leader_election.cc:343] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.506117 24044 leader_election.cc:290] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:06:54.546873 23482 leader_election.cc:304] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:54.547788 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e66e5d5789d4dc3a56f76f34b603bad" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:54.548122 24003 raft_consensus.cc:2804] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.548511 23515 raft_consensus.cc:3060] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.548707 24003 raft_consensus.cc:697] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:54.549415 23517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:54.549932 24003 consensus_queue.cc:237] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.552594 23831 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.553131 23831 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:54.555125 23831 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:54.562680 24020 consensus_queue.cc:1048] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.566870 23639 tablet_bootstrap.cc:654] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.579947 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e66e5d5789d4dc3a56f76f34b603bad" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:54.582226 23645 raft_consensus.cc:359] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.582876 23645 raft_consensus.cc:385] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.583110 23645 raft_consensus.cc:740] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.583766 23645 consensus_queue.cc:260] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.586444 23645 ts_tablet_manager.cc:1434] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.000s
I20260709 13:06:54.568792 23440 raft_consensus.cc:2468] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:54.587777 23645 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: Bootstrap starting.
W20260709 13:06:54.593956 23408 leader_election.cc:343] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.568590 23558 leader_election.cc:304] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:54.605221 23866 raft_consensus.cc:2804] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.605772 23866 raft_consensus.cc:493] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.606209 23866 raft_consensus.cc:3060] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.618206 23645 tablet_bootstrap.cc:654] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.640161 23587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:54.647081 23590 raft_consensus.cc:3060] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.662149 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:54.663709 23366 raft_consensus.cc:2393] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
W20260709 13:06:54.673899 24062 log.cc:927] Time spent T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:06:54.709201 23361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:54.709997 23361 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 0.
I20260709 13:06:54.713469 23363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:54.714159 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 93849.7us
I20260709 13:06:54.714643 23484 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:54.718411 23642 raft_consensus.cc:359] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.719450 23642 raft_consensus.cc:385] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.719779 23642 raft_consensus.cc:740] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.720330 23821 raft_consensus.cc:2804] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.720731 23821 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.721189 23821 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.722980 23866 raft_consensus.cc:515] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.715693 23363 raft_consensus.cc:2468] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 0.
W20260709 13:06:54.745925 23333 leader_election.cc:343] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.689787 24099 consensus_queue.cc:1048] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.233s
I20260709 13:06:54.754428 23642 consensus_queue.cc:260] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.755681 23333 leader_election.cc:304] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:54.748611 24091 consensus_queue.cc:1048] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:06:54.778039 23410 leader_election.cc:304] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1
I20260709 13:06:54.779634 23881 raft_consensus.cc:2804] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.772110 23866 leader_election.cc:290] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:54.769228 23482 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:54.782748 23881 raft_consensus.cc:493] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.785156 23881 raft_consensus.cc:3060] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.785897 24090 raft_consensus.cc:2749] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.786792 24117 raft_consensus.cc:493] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.787365 24117 raft_consensus.cc:515] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.780942 23516 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.795379 24117 leader_election.cc:290] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:54.805601 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:54.806329 23370 raft_consensus.cc:3060] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.810074 23642 ts_tablet_manager.cc:1434] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.291s	user 0.009s	sys 0.001s
I20260709 13:06:54.814028 23642 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:54.814792 23368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:54.815485 23368 raft_consensus.cc:2468] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:54.816699 23558 leader_election.cc:304] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:54.818862 24009 raft_consensus.cc:2804] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.819351 24009 raft_consensus.cc:493] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.819798 24009 raft_consensus.cc:3060] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:54.829023 23333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 83129.1us
I20260709 13:06:54.830469 23333 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:54.837373 24008 raft_consensus.cc:2804] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.837785 24008 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.838125 24008 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.840409 23639 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:54.841492 23639 ts_tablet_manager.cc:1403] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.506s	user 0.023s	sys 0.020s
W20260709 13:06:54.841228 23408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 69971.4us
W20260709 13:06:54.843947 23333 leader_election.cc:343] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.849939 23333 leader_election.cc:304] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e
I20260709 13:06:54.850813 23728 raft_consensus.cc:2749] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.860886 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:54.866250 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c366e593434e49a9b0f8d3a41616077d" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
W20260709 13:06:54.849347 23410 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 69647.3us
I20260709 13:06:54.874914 23590 raft_consensus.cc:2468] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
W20260709 13:06:54.877978 23556 leader_election.cc:343] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.869418 23639 raft_consensus.cc:359] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.881603 23639 raft_consensus.cc:385] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.882128 23639 raft_consensus.cc:740] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
W20260709 13:06:54.882478 23556 leader_election.cc:343] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.909893 24008 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.895792 23639 consensus_queue.cc:260] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.932127 23408 leader_election.cc:304] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
W20260709 13:06:54.935329 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 77808.4us
I20260709 13:06:54.937403 24093 raft_consensus.cc:2804] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.937969 24093 raft_consensus.cc:697] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:54.938908 24093 consensus_queue.cc:237] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:06:54.941964 24009 raft_consensus.cc:515] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:54.944334 23885 raft_consensus.cc:2749] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.895923 23515 raft_consensus.cc:2468] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:54.951108 23881 raft_consensus.cc:515] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.954095 23831 consensus_queue.cc:1048] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.962610 23881 leader_election.cc:290] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:54.966235 23370 raft_consensus.cc:2468] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:54.966508 23639 ts_tablet_manager.cc:1434] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.125s	user 0.010s	sys 0.008s
I20260709 13:06:54.982477 23821 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:54.985447 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:54.986279 23516 raft_consensus.cc:3060] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.997021 23364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64b73b48f1244e19a12b43c7f7a82cc5" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:54.997915 23364 raft_consensus.cc:3060] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.998324 23366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:54.999490 23366 raft_consensus.cc:2393] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:55.000985 24020 consensus_queue.cc:1048] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.002130 23558 leader_election.cc:304] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:55.003923 24117 raft_consensus.cc:2804] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.004572 24117 raft_consensus.cc:697] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:55.005563 24117 consensus_queue.cc:237] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.006331 23821 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:55.008106 24008 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:54.997217 24009 leader_election.cc:290] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:55.016227 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:55.016961 23445 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:55.025075 23316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:55.025350 23370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:55.026301 23370 raft_consensus.cc:3060] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.028020 23642 tablet_bootstrap.cc:654] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.032673 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:55.033318 23639 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: Bootstrap starting.
I20260709 13:06:55.041422 23364 raft_consensus.cc:2468] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:55.043547 23445 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 1.
W20260709 13:06:55.046119 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 51960us
I20260709 13:06:55.060905 23515 raft_consensus.cc:2393] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d60ed2fc1ba64596942b6df1bf93b4ca in current term 1: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:55.063915 23516 raft_consensus.cc:2468] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:55.070021 23370 raft_consensus.cc:2468] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:55.113890 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:55.119596 23410 leader_election.cc:304] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:55.125169 24093 raft_consensus.cc:2804] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.125820 24093 raft_consensus.cc:697] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:55.126843 24093 consensus_queue.cc:237] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:55.099731 23645 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: No bootstrap required, opened a new log
I20260709 13:06:55.129338 23645 ts_tablet_manager.cc:1403] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: Time spent bootstrapping tablet: real 0.542s	user 0.022s	sys 0.041s
I20260709 13:06:55.130616 23333 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
W20260709 13:06:55.136749 23556 leader_election.cc:343] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.153489 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:55.154583 23443 raft_consensus.cc:2393] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 1: Already voted for candidate d60ed2fc1ba64596942b6df1bf93b4ca in this term.
I20260709 13:06:55.158756 23482 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:55.160888 24089 raft_consensus.cc:2749] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.168536 24113 raft_consensus.cc:2804] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.169245 24113 raft_consensus.cc:697] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:06:55.170359 24113 consensus_queue.cc:237] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:55.171813 24123 consensus_queue.cc:1048] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.140784 23558 leader_election.cc:304] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:06:55.184664 24092 raft_consensus.cc:2804] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:55.184793 23558 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 48435.5us
I20260709 13:06:55.185407 24092 raft_consensus.cc:697] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:55.192287 24092 consensus_queue.cc:237] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.238001 23642 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:55.239118 23642 ts_tablet_manager.cc:1403] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.425s	user 0.023s	sys 0.004s
I20260709 13:06:55.246259 23979 consensus_queue.cc:1048] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:06:55.255849 23639 tablet_bootstrap.cc:654] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.260350 23645 raft_consensus.cc:359] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.262261 23645 raft_consensus.cc:385] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.262629 23645 raft_consensus.cc:740] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.279047 23645 consensus_queue.cc:260] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.282776 23645 ts_tablet_manager.cc:1434] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: Time spent starting tablet: real 0.153s	user 0.008s	sys 0.000s
I20260709 13:06:55.309799 23268 catalog_manager.cc:5697] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.367079 23639 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: No bootstrap required, opened a new log
I20260709 13:06:55.367690 23639 ts_tablet_manager.cc:1403] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: Time spent bootstrapping tablet: real 0.335s	user 0.023s	sys 0.035s
I20260709 13:06:55.369560 23642 raft_consensus.cc:359] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.370311 23642 raft_consensus.cc:385] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.370616 23642 raft_consensus.cc:740] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.371337 23642 consensus_queue.cc:260] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.374145 23639 raft_consensus.cc:359] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.374855 23639 raft_consensus.cc:385] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.375129 23639 raft_consensus.cc:740] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.375890 23639 consensus_queue.cc:260] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.376994 23642 ts_tablet_manager.cc:1434] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.137s	user 0.007s	sys 0.001s
I20260709 13:06:55.378055 23642 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: Bootstrap starting.
I20260709 13:06:55.378252 23639 ts_tablet_manager.cc:1434] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:06:55.387732 23642 tablet_bootstrap.cc:654] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.389894 24093 consensus_queue.cc:1048] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.415256 23440 raft_consensus.cc:3060] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.423421 23642 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: No bootstrap required, opened a new log
I20260709 13:06:55.422960 23267 catalog_manager.cc:5697] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.423991 23642 ts_tablet_manager.cc:1403] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent bootstrapping tablet: real 0.046s	user 0.003s	sys 0.027s
I20260709 13:06:55.431138 23642 raft_consensus.cc:359] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.431936 23642 raft_consensus.cc:385] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.432236 23642 raft_consensus.cc:740] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.433142 23642 consensus_queue.cc:260] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.435353 23642 ts_tablet_manager.cc:1434] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:06:55.433770 23267 catalog_manager.cc:5697] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.437067 24092 consensus_queue.cc:1048] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.441229 24093 consensus_queue.cc:1048] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.462178 23866 consensus_queue.cc:1048] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.467607 23265 catalog_manager.cc:5697] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.495405 23866 raft_consensus.cc:493] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.495947 23866 raft_consensus.cc:515] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.496011 23265 catalog_manager.cc:5697] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 1 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.499234 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:55.499622 23364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:55.500051 23516 raft_consensus.cc:2468] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:55.500379 23364 raft_consensus.cc:2468] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 0.
I20260709 13:06:55.501885 23558 leader_election.cc:304] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:55.498216 23866 leader_election.cc:290] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:55.502744 24091 raft_consensus.cc:2804] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.503126 24091 raft_consensus.cc:493] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.503484 24091 raft_consensus.cc:3060] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.511125 24091 raft_consensus.cc:515] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.514297 23364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:55.514928 23364 raft_consensus.cc:3060] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.525014 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787a4af7968d4c359c9efbc9abfe9b40" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:55.525707 23516 raft_consensus.cc:3060] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.532147 23364 raft_consensus.cc:2468] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:55.538112 23558 leader_election.cc:304] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:55.532986 24091 leader_election.cc:290] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 1 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:55.535492 24093 consensus_queue.cc:1048] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.536192 23516 raft_consensus.cc:2468] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:55.538961 23866 raft_consensus.cc:2804] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.539503 23866 raft_consensus.cc:697] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:55.540375 23866 consensus_queue.cc:237] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:55.548864 23265 catalog_manager.cc:5697] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.557988 23882 consensus_queue.cc:1048] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.589195 23265 catalog_manager.cc:5697] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 1 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.623764 23882 raft_consensus.cc:493] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.624313 23882 raft_consensus.cc:515] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:55.626611 23882 leader_election.cc:290] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:55.627377 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:55.627573 23363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce891b8160664205892014cec10858ff" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:55.628459 23363 raft_consensus.cc:2393] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 1: Already voted for candidate ceb56da8fd5145648a898f5487219c6e in this term.
I20260709 13:06:55.629513 23410 leader_election.cc:304] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:55.630280 23882 raft_consensus.cc:3060] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.638401 23882 raft_consensus.cc:2749] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.685469 24113 consensus_queue.cc:1048] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.698243 23866 consensus_queue.cc:1048] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.711911 24023 consensus_queue.cc:1048] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.714902 23866 consensus_queue.cc:1048] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.131886 24091 consensus_queue.cc:1048] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.148340 24091 consensus_queue.cc:1048] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.339155 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
W20260709 13:06:57.341296 23484 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:06:57.350875 23484 consensus_peers.cc:597] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.353682 23484 consensus_peers.cc:597] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.381958 23558 consensus_peers.cc:597] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.384490 23558 consensus_peers.cc:597] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.388510 23410 consensus_peers.cc:597] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.389493 23410 consensus_peers.cc:597] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.395766 23410 consensus_peers.cc:597] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.404356 23558 consensus_peers.cc:597] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.425686 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:06:57.426836 23558 consensus_peers.cc:597] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.426877 13967 tablet_replica.cc:333] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.428349 13967 raft_consensus.cc:2243] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.428941 13967 raft_consensus.cc:2272] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.430902 13967 tablet_replica.cc:333] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.431496 13967 raft_consensus.cc:2243] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.431908 13967 raft_consensus.cc:2272] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.433756 13967 tablet_replica.cc:333] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.434441 13967 raft_consensus.cc:2243] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.434877 13967 raft_consensus.cc:2272] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.437518 13967 tablet_replica.cc:333] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
W20260709 13:06:57.437867 23558 consensus_peers.cc:597] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.438320 13967 raft_consensus.cc:2243] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.438827 13967 raft_consensus.cc:2272] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.440738 13967 tablet_replica.cc:333] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.441412 13967 raft_consensus.cc:2243] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.441818 13967 raft_consensus.cc:2272] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.443806 13967 tablet_replica.cc:333] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.444692 13967 raft_consensus.cc:2243] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.445180 13967 raft_consensus.cc:2272] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.447546 13967 tablet_replica.cc:333] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.448148 13967 raft_consensus.cc:2243] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.449172 13967 raft_consensus.cc:2272] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.452494 13967 tablet_replica.cc:333] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.453493 13967 raft_consensus.cc:2243] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.454550 13967 raft_consensus.cc:2272] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.457443 13967 tablet_replica.cc:333] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.458205 13967 raft_consensus.cc:2243] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.458694 13967 raft_consensus.cc:2272] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.461243 13967 tablet_replica.cc:333] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.461822 13967 raft_consensus.cc:2243] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.462278 13967 raft_consensus.cc:2272] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.464432 13967 tablet_replica.cc:333] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.465116 13967 raft_consensus.cc:2243] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.465988 13967 raft_consensus.cc:2272] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.468514 13967 tablet_replica.cc:333] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.469354 13967 raft_consensus.cc:2243] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.470000 13967 raft_consensus.cc:2272] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.472306 13967 tablet_replica.cc:333] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.473109 13967 raft_consensus.cc:2243] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.473650 13967 raft_consensus.cc:2272] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.476547 13967 tablet_replica.cc:333] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.477552 13967 raft_consensus.cc:2243] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.478653 13967 raft_consensus.cc:2272] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:57.479758 23558 consensus_peers.cc:597] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.481683 13967 tablet_replica.cc:333] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.482527 13967 raft_consensus.cc:2243] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:57.483065 23484 consensus_peers.cc:597] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.483527 13967 raft_consensus.cc:2272] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.486008 13967 tablet_replica.cc:333] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.486858 13967 raft_consensus.cc:2243] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.487823 13967 raft_consensus.cc:2272] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.491047 13967 tablet_replica.cc:333] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.491858 13967 raft_consensus.cc:2243] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.492334 13967 raft_consensus.cc:2272] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.495008 13967 tablet_replica.cc:333] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.495950 13967 raft_consensus.cc:2243] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.496500 13967 raft_consensus.cc:2272] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.499248 13967 tablet_replica.cc:333] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.500005 13967 raft_consensus.cc:2243] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.501214 13967 raft_consensus.cc:2272] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.503715 13967 tablet_replica.cc:333] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.504410 13967 raft_consensus.cc:2243] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.504920 13967 raft_consensus.cc:2272] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.506992 13967 tablet_replica.cc:333] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.507598 13967 raft_consensus.cc:2243] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.508111 13967 raft_consensus.cc:2272] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.510273 13967 tablet_replica.cc:333] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.510869 13967 raft_consensus.cc:2243] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.511646 13967 raft_consensus.cc:2272] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.514003 13967 tablet_replica.cc:333] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.514626 13967 raft_consensus.cc:2243] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.515058 13967 raft_consensus.cc:2272] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:57.517124 23410 consensus_peers.cc:597] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.517802 13967 tablet_replica.cc:333] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.518579 13967 raft_consensus.cc:2243] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:57.519009 23558 consensus_peers.cc:597] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.519158 13967 raft_consensus.cc:2272] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.521921 13967 tablet_replica.cc:333] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.522619 13967 raft_consensus.cc:2243] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.523387 13967 raft_consensus.cc:2272] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.525446 13967 tablet_replica.cc:333] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.525928 13967 raft_consensus.cc:2243] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.526329 13967 raft_consensus.cc:2272] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.528163 13967 tablet_replica.cc:333] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.528649 13967 raft_consensus.cc:2243] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.529083 13967 raft_consensus.cc:2272] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.530884 13967 tablet_replica.cc:333] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.531352 13967 raft_consensus.cc:2243] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.532064 13967 raft_consensus.cc:2272] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.533967 13967 tablet_replica.cc:333] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.534528 13967 raft_consensus.cc:2243] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.535157 13967 raft_consensus.cc:2272] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.538043 13967 tablet_replica.cc:333] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.538800 13967 raft_consensus.cc:2243] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:57.539533 23558 consensus_peers.cc:597] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.540105 13967 raft_consensus.cc:2272] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.543251 13967 tablet_replica.cc:333] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.543910 13967 raft_consensus.cc:2243] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.544440 13967 raft_consensus.cc:2272] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.546581 13967 tablet_replica.cc:333] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.547245 13967 raft_consensus.cc:2243] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.547716 13967 raft_consensus.cc:2272] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.550123 13967 tablet_replica.cc:333] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.550877 13967 raft_consensus.cc:2243] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.551486 13967 raft_consensus.cc:2272] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:57.553649 23410 consensus_peers.cc:597] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.554471 13967 tablet_replica.cc:333] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.555150 13967 raft_consensus.cc:2243] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.555966 13967 raft_consensus.cc:2272] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.558167 13967 tablet_replica.cc:333] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.558765 13967 raft_consensus.cc:2243] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.559238 13967 raft_consensus.cc:2272] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.561832 13967 tablet_replica.cc:333] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.562645 13967 raft_consensus.cc:2243] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.563274 13967 raft_consensus.cc:2272] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.566136 13967 tablet_replica.cc:333] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.566891 13967 raft_consensus.cc:2243] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.567962 13967 raft_consensus.cc:2272] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.570973 13967 tablet_replica.cc:333] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.571849 13967 raft_consensus.cc:2243] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.572479 13967 raft_consensus.cc:2272] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.575459 13967 tablet_replica.cc:333] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.576166 13967 raft_consensus.cc:2243] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.576725 13967 raft_consensus.cc:2272] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.579020 13967 tablet_replica.cc:333] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.579650 13967 raft_consensus.cc:2243] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.580123 13967 raft_consensus.cc:2272] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.582213 13967 tablet_replica.cc:333] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.582783 13967 raft_consensus.cc:2243] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:57.583794 13967 raft_consensus.cc:2272] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.586685 13967 tablet_replica.cc:333] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.587579 13967 raft_consensus.cc:2243] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:06:57.587842 23484 consensus_peers.cc:597] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.588789 13967 raft_consensus.cc:2272] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.591251 13967 tablet_replica.cc:333] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.591933 13967 raft_consensus.cc:2243] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.592509 13967 raft_consensus.cc:2272] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.595330 13967 tablet_replica.cc:333] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.595892 13967 raft_consensus.cc:2243] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.596305 13967 raft_consensus.cc:2272] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.598574 13967 tablet_replica.cc:333] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: stopping tablet replica
I20260709 13:06:57.599046 13967 raft_consensus.cc:2243] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.599427 13967 raft_consensus.cc:2272] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.667740 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
W20260709 13:06:57.698136 23410 consensus_peers.cc:597] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.744433 23484 consensus_peers.cc:597] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.753288 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:57.756520 23558 consensus_peers.cc:597] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.774345 24167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:57.778414 23558 consensus_peers.cc:597] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.782811 23410 consensus_peers.cc:597] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.783679 23410 consensus_peers.cc:597] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.783640 24168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:57.786504 24170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:57.786762 13967 server_base.cc:1061] running on GCE node
I20260709 13:06:57.791597 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:57.791966 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:57.792248 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602417792226 us; error 0 us; skew 500 ppm
I20260709 13:06:57.793068 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:06:57.794464 23558 consensus_peers.cc:597] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.799417 23410 consensus_peers.cc:597] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.810760 23484 consensus_peers.cc:597] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.815165 13967 webserver.cc:533] Webserver started at http://127.13.163.193:34243/ using document root <none> and password file <none>
I20260709 13:06:57.815828 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:57.816061 13967 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:06:57.818513 23558 consensus_peers.cc:597] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:57.824108 13967 fs_manager.cc:714] Time spent opening directory manager: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:06:57.841135 24176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:57.842243 13967 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:57.842677 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
format_stamp: "Formatted at 2026-07-09 13:06:44 on dist-test-slave-w3b1"
I20260709 13:06:57.843254 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:57.879276 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:57.881034 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:57.884514 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:57.890354 24183 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:06:57.898679 23558 consensus_peers.cc:597] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.929940 23410 consensus_peers.cc:597] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:57.953512 23558 consensus_peers.cc:597] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:58.156170 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:06:58.156611 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.267s	user 0.007s	sys 0.000s
I20260709 13:06:58.156997 13967 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:06:58.164198 24183 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.193182 24183 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.194213 24183 tablet_bootstrap.cc:492] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.195132 24183 ts_tablet_manager.cc:1403] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.011s
I20260709 13:06:58.197746 24183 raft_consensus.cc:359] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.198491 24183 raft_consensus.cc:740] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.206102 24183 consensus_queue.cc:260] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.215850 24183 ts_tablet_manager.cc:1434] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.005s
I20260709 13:06:58.216763 24183 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.248404 24183 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.249387 24183 tablet_bootstrap.cc:492] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.250001 24183 ts_tablet_manager.cc:1403] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.001s
I20260709 13:06:58.252802 24183 raft_consensus.cc:359] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.253454 24183 raft_consensus.cc:740] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.254079 24183 consensus_queue.cc:260] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.255434 24183 ts_tablet_manager.cc:1434] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:06:58.256035 24183 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.268173 24183 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.269301 24183 tablet_bootstrap.cc:492] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.269937 24183 ts_tablet_manager.cc:1403] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.007s
I20260709 13:06:58.272291 24183 raft_consensus.cc:359] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.273052 24183 raft_consensus.cc:740] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.273797 24183 consensus_queue.cc:260] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.277277 24183 ts_tablet_manager.cc:1434] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:58.278087 24183 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.328517 24183 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.329442 24183 tablet_bootstrap.cc:492] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.330072 24183 ts_tablet_manager.cc:1403] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.052s	user 0.033s	sys 0.016s
I20260709 13:06:58.332502 24183 raft_consensus.cc:359] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.333175 24183 raft_consensus.cc:740] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.333812 24183 consensus_queue.cc:260] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.336127 24183 ts_tablet_manager.cc:1434] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:58.336906 24183 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.351531 24183 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.352442 24183 tablet_bootstrap.cc:492] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.353060 24183 ts_tablet_manager.cc:1403] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:06:58.355376 24183 raft_consensus.cc:359] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.356000 24183 raft_consensus.cc:740] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.356643 24183 consensus_queue.cc:260] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.358491 24183 ts_tablet_manager.cc:1434] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:58.359211 24183 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.385779 24183 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.386778 24183 tablet_bootstrap.cc:492] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.387465 24183 ts_tablet_manager.cc:1403] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.015s
I20260709 13:06:58.389963 24183 raft_consensus.cc:359] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.390661 24183 raft_consensus.cc:740] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.391360 24183 consensus_queue.cc:260] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.393246 24183 ts_tablet_manager.cc:1434] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:58.394029 24183 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.420094 24183 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.421433 24183 tablet_bootstrap.cc:492] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.421991 24183 ts_tablet_manager.cc:1403] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
I20260709 13:06:58.423308 13967 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:06:58.423668 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.267s	user 0.252s	sys 0.000s
I20260709 13:06:58.424036 24183 raft_consensus.cc:359] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.424583 24183 raft_consensus.cc:740] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.425237 24183 consensus_queue.cc:260] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.426787 24183 ts_tablet_manager.cc:1434] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:06:58.427455 24183 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.450610 24183 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.451470 24183 tablet_bootstrap.cc:492] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.452114 24183 ts_tablet_manager.cc:1403] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.004s
I20260709 13:06:58.454403 24183 raft_consensus.cc:359] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.455143 24183 raft_consensus.cc:740] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.455950 24183 consensus_queue.cc:260] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.458024 24183 ts_tablet_manager.cc:1434] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:58.458846 24183 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.498739 24183 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.499787 24183 tablet_bootstrap.cc:492] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.500550 24183 ts_tablet_manager.cc:1403] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.015s
I20260709 13:06:58.502952 24183 raft_consensus.cc:359] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.503511 24183 raft_consensus.cc:740] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.504159 24183 consensus_queue.cc:260] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.505740 24183 ts_tablet_manager.cc:1434] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:58.506405 24183 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.513782 24158 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.514394 24158 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.517802 24120 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.518543 24120 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.522495 24120 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.527678 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.529215 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
W20260709 13:06:58.529765 23410 leader_election.cc:336] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.535825 23408 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.536826 24158 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.537847 24186 raft_consensus.cc:2749] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:58.539249 23484 leader_election.cc:336] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.539712 23484 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.542981 24158 raft_consensus.cc:2749] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.543926 24183 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.544979 24183 tablet_bootstrap.cc:492] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.545650 24183 ts_tablet_manager.cc:1403] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.004s
I20260709 13:06:58.548085 24183 raft_consensus.cc:359] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.548767 24183 raft_consensus.cc:740] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.549552 24183 consensus_queue.cc:260] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.550629 24173 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.551658 24183 ts_tablet_manager.cc:1434] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:58.551328 24173 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.552492 24183 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.557554 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.565379 24173 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:06:58.568928 23558 leader_election.cc:336] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.569592 24183 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.569516 23558 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.570653 24183 tablet_bootstrap.cc:492] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.570832 24173 raft_consensus.cc:2749] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.574429 24173 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.574998 24173 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.576210 24183 ts_tablet_manager.cc:1403] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.003s
I20260709 13:06:58.580508 24120 raft_consensus.cc:493] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.581079 24120 raft_consensus.cc:515] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.581854 24183 raft_consensus.cc:359] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.582551 24183 raft_consensus.cc:740] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.583180 24120 leader_election.cc:290] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.583369 24183 consensus_queue.cc:260] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.583727 24120 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.584233 24120 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.586308 24120 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.587093 24158 raft_consensus.cc:493] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.587849 24158 raft_consensus.cc:515] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.588388 24183 ts_tablet_manager.cc:1434] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:06:58.589363 24183 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.594224 24185 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.595072 23592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:58.594836 24185 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.597723 24185 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.598696 24185 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.599135 24185 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
W20260709 13:06:58.600121 23558 leader_election.cc:336] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.601505 24173 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:58.603274 23410 leader_election.cc:336] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:06:58.603835 23410 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.605202 23408 leader_election.cc:304] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.608181 24120 raft_consensus.cc:2749] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.609632 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.606315 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:58.606841 24185 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.612702 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.618947 23515 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:58.614470 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.620138 23439 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:58.615347 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.623150 23408 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.624199 24120 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.624575 24120 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.624966 24120 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:58.625268 23484 leader_election.cc:336] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.625507 23556 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.617024 24183 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.627373 24091 raft_consensus.cc:2749] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.628206 24183 tablet_bootstrap.cc:492] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.629102 24183 ts_tablet_manager.cc:1403] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.003s
I20260709 13:06:58.631865 24183 raft_consensus.cc:359] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.632613 24183 raft_consensus.cc:740] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.633476 24183 consensus_queue.cc:260] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.635742 24183 ts_tablet_manager.cc:1434] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:06:58.636538 24183 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.637439 23482 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:58.640192 24185 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.640592 24185 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.640995 24185 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.645227 24186 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.645766 24186 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.647848 24186 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.648376 23482 leader_election.cc:304] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.649446 24184 raft_consensus.cc:2749] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:58.653704 23410 leader_election.cc:336] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.655525 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.657243 23408 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.658227 24186 raft_consensus.cc:2749] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.662392 24186 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.663028 24186 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.673767 24173 raft_consensus.cc:493] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.620383 24158 leader_election.cc:290] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.674415 24173 raft_consensus.cc:515] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.676102 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:06:58.682253 23410 leader_election.cc:336] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.682698 23410 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.665715 24186 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.683653 24105 raft_consensus.cc:2749] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.684062 24158 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
W20260709 13:06:58.684329 23484 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.691787 23866 raft_consensus.cc:493] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.693034 24249 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
W20260709 13:06:58.693089 23484 leader_election.cc:336] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.693847 23484 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.693616 24249 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.697896 24249 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.698521 24249 raft_consensus.cc:2749] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.699440 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.699859 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.700191 23515 raft_consensus.cc:2468] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:58.695189 24158 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.702219 23556 leader_election.cc:304] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:58.703332 24091 raft_consensus.cc:2804] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.703809 24091 raft_consensus.cc:493] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.704208 24091 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.704208 24173 leader_election.cc:290] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.708001 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.711385 23439 raft_consensus.cc:2468] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:58.716677 23482 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:58.726864 24184 raft_consensus.cc:2804] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.682550 24120 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.731702 24184 raft_consensus.cc:493] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.732394 24184 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.692058 24185 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.734862 24185 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.695299 23866 raft_consensus.cc:515] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.703387 24158 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:58.727142 23558 leader_election.cc:336] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.720289 24183 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.740121 24183 tablet_bootstrap.cc:492] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.746217 24186 raft_consensus.cc:493] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.748420 24186 raft_consensus.cc:515] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
W20260709 13:06:58.748668 23484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 55610.5us
I20260709 13:06:58.750545 24186 leader_election.cc:290] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:06:58.746230 23558 leader_election.cc:336] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.751631 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:58.747676 24091 raft_consensus.cc:515] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.754103 23866 leader_election.cc:290] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:58.747056 23410 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.759007 24091 leader_election.cc:290] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.740808 24183 ts_tablet_manager.cc:1403] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.104s	user 0.024s	sys 0.007s
W20260709 13:06:58.763643 23558 leader_election.cc:336] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:06:58.763397 23484 leader_election.cc:336] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:06:58.766530 23484 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.742481 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:58.769901 24183 raft_consensus.cc:359] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.770450 24183 raft_consensus.cc:740] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.771051 24183 consensus_queue.cc:260] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.771242 24158 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.772701 24183 ts_tablet_manager.cc:1434] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 13:06:58.775432 24183 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.762095 24120 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.778577 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:58.779929 23515 raft_consensus.cc:2393] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 643f0d55f5ac4308a9f1d5e7158b43fe in current term 2: Already voted for candidate 12589c3399b641fc80f39058a475f8a1 in this term.
I20260709 13:06:58.781766 23408 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.772104 24158 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.786180 24158 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.788254 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.787828 23441 raft_consensus.cc:2393] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12589c3399b641fc80f39058a475f8a1 in current term 2: Already voted for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in this term.
I20260709 13:06:58.791170 24120 raft_consensus.cc:2749] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:06:58.797186 23410 leader_election.cc:336] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.809343 23410 leader_election.cc:304] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.795990 23482 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.813616 24105 raft_consensus.cc:2749] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.815992 24185 raft_consensus.cc:2749] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.816708 24184 raft_consensus.cc:515] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.820442 24184 leader_election.cc:290] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:58.821660 23484 leader_election.cc:336] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.822379 23484 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.823329 24184 raft_consensus.cc:2749] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:58.825007 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 80174.6us
I20260709 13:06:58.833683 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be3f330957084b66847db5f71ee7bc36" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:58.834318 23516 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.835922 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6c9e6d5c9540229369e18bc85a6a0d" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:58.836578 23441 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.836864 24183 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.837980 24183 tablet_bootstrap.cc:492] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.838655 24183 ts_tablet_manager.cc:1403] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.019s
I20260709 13:06:58.841579 24105 raft_consensus.cc:493] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.842231 24105 raft_consensus.cc:515] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.842708 24183 raft_consensus.cc:359] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.843487 24183 raft_consensus.cc:740] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.844403 24183 consensus_queue.cc:260] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.845697 23441 raft_consensus.cc:2468] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:06:58.848024 23482 leader_election.cc:304] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:58.851644 23516 raft_consensus.cc:2468] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
I20260709 13:06:58.849624 24158 raft_consensus.cc:2804] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.853768 24183 ts_tablet_manager.cc:1434] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:06:58.857292 24183 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.853377 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.855960 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.861814 24186 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.864199 24186 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.866405 24186 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.867530 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:58.869042 23590 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:58.870621 23408 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:58.871779 24186 raft_consensus.cc:2804] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.873886 24186 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.855041 24105 leader_election.cc:290] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.854118 24158 raft_consensus.cc:697] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:58.875250 24186 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.876183 24158 consensus_queue.cc:237] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.810079 24253 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.861668 23556 leader_election.cc:304] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.881469 24183 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:58.882306 24105 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.882529 24183 tablet_bootstrap.cc:492] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:58.883426 24183 ts_tablet_manager.cc:1403] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.003s
I20260709 13:06:58.883939 24091 raft_consensus.cc:2749] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.883028 24105 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.886581 24105 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.886406 24183 raft_consensus.cc:359] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.887326 24183 raft_consensus.cc:740] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.888195 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.888230 24183 consensus_queue.cc:260] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.889319 23515 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:58.891278 24183 ts_tablet_manager.cc:1434] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:58.879419 24253 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.891461 23408 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:58.892194 24183 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.892513 24120 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.893662 24256 raft_consensus.cc:493] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.892925 24091 raft_consensus.cc:493] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.894722 24091 raft_consensus.cc:515] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.894654 24256 raft_consensus.cc:515] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.898475 24256 leader_election.cc:290] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.899719 24091 leader_election.cc:290] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:58.875957 23484 leader_election.cc:336] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.900771 23484 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:58.893682 24120 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.902016 24249 raft_consensus.cc:2749] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.892323 23556 leader_election.cc:304] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
W20260709 13:06:58.917754 23484 leader_election.cc:336] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.903851 24120 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:58.913192 24173 raft_consensus.cc:2804] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.926236 24173 raft_consensus.cc:697] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:58.927413 24173 consensus_queue.cc:237] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:58.907449 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:58.926019 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:58.929250 23515 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:58.929373 23589 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:58.931279 24120 raft_consensus.cc:493] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.931879 24120 raft_consensus.cc:515] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:58.914059 24105 raft_consensus.cc:2804] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.904413 24253 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:58.939000 23482 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
W20260709 13:06:58.941918 23558 leader_election.cc:336] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:06:58.943670 23558 leader_election.cc:336] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.942770 24249 raft_consensus.cc:2804] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.939783 24105 raft_consensus.cc:493] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.951063 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.952684 23444 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:58.949694 24249 raft_consensus.cc:493] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.954612 23408 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:58.954864 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:35013
I20260709 13:06:58.950937 24105 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.955013 24266 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:35013 every 8 connection(s)
W20260709 13:06:58.962975 23410 leader_election.cc:336] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.963606 23410 leader_election.cc:304] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
W20260709 13:06:58.971647 23484 leader_election.cc:336] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:06:58.958473 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:58.973246 24263 raft_consensus.cc:2804] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.973429 23440 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:58.973779 24263 raft_consensus.cc:493] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.974310 24263 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.975375 24158 raft_consensus.cc:493] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:58.976022 24158 raft_consensus.cc:515] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:58.977644 23266 catalog_manager.cc:5697] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.979259 24158 leader_election.cc:290] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:58.956499 24249 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.986745 24183 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.009359 24262 raft_consensus.cc:2749] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:59.013407 24105 raft_consensus.cc:515] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.015862 24105 leader_election.cc:290] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:59.019289 24120 leader_election.cc:290] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:59.027156 24183 tablet_bootstrap.cc:492] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.028450 24183 ts_tablet_manager.cc:1403] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.136s	user 0.022s	sys 0.016s
I20260709 13:06:59.031365 24183 raft_consensus.cc:359] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.030944 23268 catalog_manager.cc:5697] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 2 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.032069 24183 raft_consensus.cc:740] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.033046 24183 consensus_queue.cc:260] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.041851 24183 ts_tablet_manager.cc:1434] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:59.043453 24183 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:58.999523 24184 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.044740 24184 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.046830 24184 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:06:59.048346 23408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:36741, user_credentials={real_user=slave}} blocked reactor thread for 92247us
W20260709 13:06:59.049062 23482 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 68484.9us
I20260709 13:06:59.050073 23556 leader_election.cc:304] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.047617 24249 raft_consensus.cc:515] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:59.051935 24173 raft_consensus.cc:2804] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.055936 24173 raft_consensus.cc:493] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.056407 24173 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.060959 24263 raft_consensus.cc:515] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.068908 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
I20260709 13:06:59.069967 23590 raft_consensus.cc:2468] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:59.063640 23556 leader_election.cc:304] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.067087 24256 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
W20260709 13:06:59.071944 23410 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 108996us
I20260709 13:06:59.071954 23866 raft_consensus.cc:2804] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.026433 24186 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.072814 23866 raft_consensus.cc:493] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.073372 23866 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.072570 24256 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.076232 24256 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.081527 24186 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:59.083494 24263 leader_election.cc:290] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:59.086007 23866 raft_consensus.cc:515] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.093395 23866 leader_election.cc:290] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.102700 24249 leader_election.cc:290] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:59.103587 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:59.104496 23589 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.106891 24173 raft_consensus.cc:515] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.115703 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:59.119424 23515 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.111862 24186 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.129642 24272 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.130417 24272 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.130769 24186 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.133090 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:59.133801 23441 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.134914 24228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.136487 24227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.136755 23589 raft_consensus.cc:2468] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:06:59.137064 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:59.132710 24272 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
W20260709 13:06:59.139510 23484 leader_election.cc:343] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.139966 24173 leader_election.cc:290] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.139446 23444 raft_consensus.cc:2468] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 1.
I20260709 13:06:59.137548 24226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538d207f493d4b14b97bce7658551672" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.140679 24227 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:59.133196 24186 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
W20260709 13:06:59.145961 23484 leader_election.cc:343] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.146998 23484 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:06:59.156358 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:59.157243 23439 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:59.158968 23482 leader_election.cc:304] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.159798 24184 raft_consensus.cc:2804] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.160319 24184 raft_consensus.cc:697] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:59.161226 24184 consensus_queue.cc:237] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:59.163305 23515 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.171936 24249 raft_consensus.cc:2804] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.173040 24249 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.174732 23408 leader_election.cc:304] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:59.171460 23556 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:06:59.175019 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:59.177063 23516 raft_consensus.cc:2468] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:59.177462 23866 raft_consensus.cc:2804] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.178035 23866 raft_consensus.cc:493] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.178476 23866 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.173756 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:59.180315 24272 raft_consensus.cc:2804] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.180521 23589 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.180788 24272 raft_consensus.cc:493] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.181389 24272 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.182400 23408 leader_election.cc:304] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:59.183403 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1" is_pre_election: true
I20260709 13:06:59.184228 23516 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:59.189152 24264 raft_consensus.cc:2804] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.189721 24264 raft_consensus.cc:697] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.190629 24264 consensus_queue.cc:237] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.194149 23589 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.198577 23439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:59.199411 23439 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.203825 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.206328 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.185000 24249 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.196115 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:06:59.209151 23408 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:59.215660 23866 raft_consensus.cc:515] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.217796 23408 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:59.219019 24120 raft_consensus.cc:2804] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.219394 24120 raft_consensus.cc:493] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.219609 24120 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.213151 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57c381a872f4c7aae7b11e12cc439b8" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.221199 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.221367 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4bb4f04978d43798dc3c00c13d510be" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.199294 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
W20260709 13:06:59.224581 23558 leader_election.cc:343] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.226310 23408 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
W20260709 13:06:59.228227 23558 leader_election.cc:343] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.230430 24262 raft_consensus.cc:2804] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.230949 24262 raft_consensus.cc:697] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.231806 24262 consensus_queue.cc:237] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.223907 24186 raft_consensus.cc:2804] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.233088 24186 raft_consensus.cc:493] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.233139 24225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.233536 24186 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.233901 24225 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
I20260709 13:06:59.238487 23866 leader_election.cc:290] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.237396 23439 raft_consensus.cc:2468] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
W20260709 13:06:59.238869 23558 leader_election.cc:343] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.242412 24223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18748afbb84547cba3ae58d7476a2119" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.261417 24219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.228773 23515 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.275624 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.224040 23444 raft_consensus.cc:2468] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:06:59.226328 23441 raft_consensus.cc:2468] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
W20260709 13:06:59.279753 23408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.196:41791, user_credentials={real_user=slave}} blocked reactor thread for 53533.8us
I20260709 13:06:59.277114 24219 raft_consensus.cc:3060] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.277633 24223 raft_consensus.cc:3060] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:59.290482 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 82582.5us
I20260709 13:06:59.299690 23515 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.313409 24222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
W20260709 13:06:59.315495 23410 leader_election.cc:343] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:59.335352 23410 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.293378 24227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7dba7096144600b277ae19d383145e" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.319763 24228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.319869 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:06:59.340662 24221 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 1.
W20260709 13:06:59.322408 23558 leader_election.cc:343] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.342072 24272 raft_consensus.cc:515] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.326555 23556 leader_election.cc:304] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.344295 24272 leader_election.cc:290] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:59.331596 24120 raft_consensus.cc:515] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.344962 24091 raft_consensus.cc:2804] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.345511 24091 raft_consensus.cc:697] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
W20260709 13:06:59.333725 23484 consensus_peers.cc:597] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.331166 24186 raft_consensus.cc:515] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.337275 24219 raft_consensus.cc:2468] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.348173 24120 leader_election.cc:290] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
W20260709 13:06:59.348366 23410 leader_election.cc:343] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.347412 24183 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.353556 23445 raft_consensus.cc:1275] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.355794 23408 leader_election.cc:304] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:59.357003 23516 raft_consensus.cc:1275] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:06:59.359308 23410 leader_election.cc:343] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.366290 23866 consensus_queue.cc:1048] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.368876 24223 raft_consensus.cc:2468] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.369665 24263 raft_consensus.cc:2804] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.370515 24263 raft_consensus.cc:697] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.370884 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.371605 24221 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.370548 24186 leader_election.cc:290] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:06:59.371809 24263 consensus_queue.cc:237] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.388662 23556 leader_election.cc:304] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.388011 23482 leader_election.cc:304] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.397895 23514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:59.398890 23514 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.399768 24185 raft_consensus.cc:2804] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.400341 24185 raft_consensus.cc:493] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:06:59.404114 24183 tablet_bootstrap.cc:492] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.405404 24183 ts_tablet_manager.cc:1403] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.362s	user 0.018s	sys 0.014s
I20260709 13:06:59.408742 24249 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.400803 24185 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.417414 24271 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39475
I20260709 13:06:59.417951 24271 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:59.414667 24249 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.401091 24256 raft_consensus.cc:2804] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.403107 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a01f067a6214e979fafb367a095f1c1" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:06:59.410673 23587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:06:59.424612 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0108a896781443dbaf9caef1e04e5cfa" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:59.436777 23515 raft_consensus.cc:3060] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.432260 24256 raft_consensus.cc:697] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:59.425472 24184 consensus_queue.cc:1048] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 13:06:59.464762 23556 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:37247, user_credentials={real_user=slave}} blocked reactor thread for 69960.2us
I20260709 13:06:59.406677 24091 consensus_queue.cc:237] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.455065 23587 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.453506 24271 heartbeater.cc:507] Master 127.13.163.254:39475 requested a full tablet report, sending...
I20260709 13:06:59.461870 23444 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.454818 23443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:59.478721 24224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31308be47968488d8fe1db62fc5e9d48" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
W20260709 13:06:59.489058 23558 consensus_peers.cc:597] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.482108 24221 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.495507 24219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa87a90cab0475da44d21f725d485ab" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.495601 24183 raft_consensus.cc:359] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.487030 23443 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.500986 24256 consensus_queue.cc:237] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.496717 24183 raft_consensus.cc:740] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.505836 23410 leader_election.cc:304] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
W20260709 13:06:59.507109 23410 leader_election.cc:343] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.516949 24183 consensus_queue.cc:260] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
W20260709 13:06:59.519330 23410 leader_election.cc:343] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.525007 24183 ts_tablet_manager.cc:1434] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.119s	user 0.003s	sys 0.006s
I20260709 13:06:59.533418 24183 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:59.530239 23443 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
W20260709 13:06:59.534937 23484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:35013, user_credentials={real_user=slave}} blocked reactor thread for 111338us
I20260709 13:06:59.520462 24272 raft_consensus.cc:2804] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.570911 23444 raft_consensus.cc:2468] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
I20260709 13:06:59.572697 23514 raft_consensus.cc:2468] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.572690 23556 leader_election.cc:304] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.574409 24294 raft_consensus.cc:2804] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.574613 23587 raft_consensus.cc:2468] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:06:59.575062 24294 raft_consensus.cc:697] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:06:59.576390 24294 consensus_queue.cc:237] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.578001 23265 catalog_manager.cc:5697] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.581588 23408 leader_election.cc:304] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.528708 24185 raft_consensus.cc:515] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.565107 24272 raft_consensus.cc:697] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.583294 23408 leader_election.cc:304] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.595799 24272 consensus_queue.cc:237] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.583534 24186 raft_consensus.cc:2804] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.599014 24186 raft_consensus.cc:697] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.600008 24186 consensus_queue.cc:237] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:06:59.596570 24264 raft_consensus.cc:2804] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.602519 24264 raft_consensus.cc:697] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:06:59.603680 24264 consensus_queue.cc:237] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.546218 24224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:06:59.606762 24185 leader_election.cc:290] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:06:59.609067 24219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
W20260709 13:06:59.617108 23484 leader_election.cc:343] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.614948 23482 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:06:59.637159 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8912a0f40244d6bb5ff749263ff4993" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:06:59.640012 23441 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.630682 24185 raft_consensus.cc:2804] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:06:59.651232 23484 consensus_peers.cc:597] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.654958 24185 raft_consensus.cc:697] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:59.655952 24185 consensus_queue.cc:237] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.651937 23268 catalog_manager.cc:5697] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.659039 23266 catalog_manager.cc:5697] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 2 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.681073 23266 catalog_manager.cc:5697] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 2 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.638756 24224 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.711145 23441 raft_consensus.cc:2468] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:06:59.714762 23482 leader_election.cc:304] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:06:59.716266 24299 raft_consensus.cc:2804] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.716781 24299 raft_consensus.cc:697] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:06:59.717649 24299 consensus_queue.cc:237] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.719072 23587 raft_consensus.cc:1275] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.677901 23515 raft_consensus.cc:2468] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
W20260709 13:06:59.723094 23410 consensus_peers.cc:597] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.725493 23268 catalog_manager.cc:5697] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.729632 24253 consensus_queue.cc:1048] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:06:59.738752 24221 raft_consensus.cc:1275] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.740633 24120 consensus_queue.cc:1048] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.743062 24224 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:06:59.772070 23515 raft_consensus.cc:1275] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.772444 23265 catalog_manager.cc:5697] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.782835 24264 consensus_queue.cc:1048] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:59.788578 23558 consensus_peers.cc:597] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:59.790010 24219 raft_consensus.cc:1275] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.792016 24262 consensus_queue.cc:1048] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.838416 23265 catalog_manager.cc:5697] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 2 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.727866 23268 catalog_manager.cc:5697] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.846596 23268 catalog_manager.cc:5697] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.847891 23268 catalog_manager.cc:5697] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:59.849932 23558 consensus_peers.cc:597] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:59.859232 23268 catalog_manager.cc:5697] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.861310 23516 raft_consensus.cc:1275] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.863915 24264 consensus_queue.cc:1048] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.868291 24183 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.869688 24183 tablet_bootstrap.cc:492] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.870781 24183 ts_tablet_manager.cc:1403] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.338s	user 0.020s	sys 0.001s
I20260709 13:06:59.872445 23590 raft_consensus.cc:1275] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.873456 24183 raft_consensus.cc:359] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.874193 24183 raft_consensus.cc:740] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.874707 24262 consensus_queue.cc:1048] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.876663 24183 consensus_queue.cc:260] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.879949 24183 ts_tablet_manager.cc:1434] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:59.877104 23267 ts_manager.cc:194] Re-registered known tserver with Master: d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:06:59.881014 24183 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:59.885497 23266 catalog_manager.cc:5697] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 2, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:59.887144 23558 consensus_peers.cc:597] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:59.921705 23484 consensus_peers.cc:597] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:59.931648 24183 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.932744 24183 tablet_bootstrap.cc:492] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.933459 24183 ts_tablet_manager.cc:1403] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.001s
W20260709 13:06:59.934155 23410 consensus_peers.cc:597] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:59.936116 24183 raft_consensus.cc:359] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
W20260709 13:06:59.936892 23558 consensus_peers.cc:597] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.936965 24183 raft_consensus.cc:740] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.938210 24183 consensus_queue.cc:260] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:59.942782 24183 ts_tablet_manager.cc:1434] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:59.943598 24183 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:06:59.959697 24183 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.960891 24183 tablet_bootstrap.cc:492] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.961611 24183 ts_tablet_manager.cc:1403] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:06:59.964224 24183 raft_consensus.cc:359] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.965497 24183 raft_consensus.cc:740] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.966267 24183 consensus_queue.cc:260] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:06:59.971113 24183 ts_tablet_manager.cc:1434] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:59.972018 24183 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
W20260709 13:06:59.977146 23558 consensus_peers.cc:597] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:59.989347 24183 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.990581 24183 tablet_bootstrap.cc:492] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:06:59.991389 24183 ts_tablet_manager.cc:1403] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:06:59.994297 24183 raft_consensus.cc:359] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:59.995028 24183 raft_consensus.cc:740] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.995934 24183 consensus_queue.cc:260] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:06:59.998257 24183 ts_tablet_manager.cc:1434] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:59.998886 23267 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38778
I20260709 13:06:59.999111 24183 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.003870 24271 heartbeater.cc:499] Master 127.13.163.254:39475 was elected leader, sending a full tablet report...
I20260709 13:07:00.007507 23445 raft_consensus.cc:1275] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.009050 24296 consensus_queue.cc:1048] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.015475 24183 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.019039 24219 raft_consensus.cc:3060] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.021126 24183 tablet_bootstrap.cc:492] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.021915 24183 ts_tablet_manager.cc:1403] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.007s
I20260709 13:07:00.024807 24183 raft_consensus.cc:359] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.027669 24183 raft_consensus.cc:740] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.028407 24183 consensus_queue.cc:260] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.030432 24183 ts_tablet_manager.cc:1434] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:07:00.031224 24183 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.031302 24219 raft_consensus.cc:1275] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.033285 24292 consensus_queue.cc:1048] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.046964 23445 raft_consensus.cc:1275] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.048568 24292 consensus_queue.cc:1048] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.052470 24183 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.053539 24183 tablet_bootstrap.cc:492] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.054198 24183 ts_tablet_manager.cc:1403] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.000s
W20260709 13:07:00.055600 23484 consensus_peers.cc:597] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.058085 24183 raft_consensus.cc:359] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.058769 24183 raft_consensus.cc:740] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.059414 24183 consensus_queue.cc:260] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.061072 24183 ts_tablet_manager.cc:1434] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:00.061779 24183 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.064162 23515 raft_consensus.cc:1275] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.067128 24262 consensus_queue.cc:1048] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:00.070959 23558 consensus_peers.cc:597] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.086910 23519 raft_consensus.cc:1275] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.091629 24272 consensus_queue.cc:1048] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.106586 24219 raft_consensus.cc:1275] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.109339 24263 consensus_queue.cc:1048] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.113757 23590 raft_consensus.cc:1275] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.115877 24120 consensus_queue.cc:1048] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.130945 24224 raft_consensus.cc:3060] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.131731 24183 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.132718 24183 tablet_bootstrap.cc:492] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.133396 24183 ts_tablet_manager.cc:1403] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.072s	user 0.031s	sys 0.011s
W20260709 13:07:00.134305 23410 consensus_peers.cc:597] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.135834 24183 raft_consensus.cc:359] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.136607 24183 raft_consensus.cc:740] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.137324 24183 consensus_queue.cc:260] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.138898 24183 ts_tablet_manager.cc:1434] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:00.139693 24183 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.140410 24224 raft_consensus.cc:1275] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.143388 23445 raft_consensus.cc:1275] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:00.143565 23558 consensus_peers.cc:597] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.177296 24299 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:00.180099 23410 consensus_peers.cc:597] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.202601 24224 raft_consensus.cc:1275] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.202733 23444 raft_consensus.cc:1275] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.210078 24299 consensus_queue.cc:1048] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.210212 24185 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:00.212354 23410 consensus_peers.cc:597] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.219302 24291 consensus_queue.cc:1048] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:00.221830 24183 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:00.223505 23484 consensus_peers.cc:597] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.224577 24183 tablet_bootstrap.cc:492] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.225418 24183 ts_tablet_manager.cc:1403] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.086s	user 0.032s	sys 0.023s
W20260709 13:07:00.253751 23484 consensus_peers.cc:597] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.280467 23441 raft_consensus.cc:1275] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.255812 24183 raft_consensus.cc:359] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.282724 24183 raft_consensus.cc:740] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.283548 24183 consensus_queue.cc:260] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.285597 24183 ts_tablet_manager.cc:1434] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 13:07:00.286695 24183 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.306540 24184 consensus_queue.cc:1048] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:00.328052 24183 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.329176 24183 tablet_bootstrap.cc:492] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.329907 24183 ts_tablet_manager.cc:1403] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.006s
I20260709 13:07:00.332863 24183 raft_consensus.cc:359] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.333616 24183 raft_consensus.cc:740] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.334285 24183 consensus_queue.cc:260] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.338927 24183 ts_tablet_manager.cc:1434] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:00.339787 24183 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.359899 24183 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.360827 24183 tablet_bootstrap.cc:492] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.361433 24183 ts_tablet_manager.cc:1403] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.004s
I20260709 13:07:00.363278 24183 raft_consensus.cc:359] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.364102 24183 raft_consensus.cc:740] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.364768 24183 consensus_queue.cc:260] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.366882 24183 ts_tablet_manager.cc:1434] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:07:00.371204 24183 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.408696 24183 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.409621 24183 tablet_bootstrap.cc:492] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.410188 24183 ts_tablet_manager.cc:1403] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.023s
I20260709 13:07:00.412552 24183 raft_consensus.cc:359] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.413252 24183 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.414043 24183 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.415517 24183 ts_tablet_manager.cc:1434] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:00.417235 24183 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.438352 24183 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.439339 24183 tablet_bootstrap.cc:492] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
W20260709 13:07:00.439688 23410 consensus_peers.cc:597] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:00.440146 24183 ts_tablet_manager.cc:1403] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.005s
I20260709 13:07:00.442310 24183 raft_consensus.cc:359] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.442842 24183 raft_consensus.cc:740] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.443480 24183 consensus_queue.cc:260] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.445812 24183 ts_tablet_manager.cc:1434] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:07:00.446892 24183 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.460389 24183 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.461349 24183 tablet_bootstrap.cc:492] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.462007 24183 ts_tablet_manager.cc:1403] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:07:00.464375 24183 raft_consensus.cc:359] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.465087 24183 raft_consensus.cc:740] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.465654 24183 consensus_queue.cc:260] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.467631 24183 ts_tablet_manager.cc:1434] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:07:00.468916 24183 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.482349 24183 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.483129 24183 tablet_bootstrap.cc:492] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.483737 24183 ts_tablet_manager.cc:1403] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:07:00.486295 24183 raft_consensus.cc:359] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.486907 24183 raft_consensus.cc:740] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.487573 24183 consensus_queue.cc:260] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.489585 24183 ts_tablet_manager.cc:1434] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:00.490721 24183 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.505851 24183 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.506664 24183 tablet_bootstrap.cc:492] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.507339 24183 ts_tablet_manager.cc:1403] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:07:00.509881 24183 raft_consensus.cc:359] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.510603 24183 raft_consensus.cc:740] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.511230 24183 consensus_queue.cc:260] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.513180 24183 ts_tablet_manager.cc:1434] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:00.515532 24219 raft_consensus.cc:3060] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.517408 24183 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.526912 24221 raft_consensus.cc:3060] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.544169 24183 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.545305 24183 tablet_bootstrap.cc:492] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.546274 24183 ts_tablet_manager.cc:1403] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.010s
I20260709 13:07:00.548733 24183 raft_consensus.cc:359] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.549419 24183 raft_consensus.cc:740] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.550133 24183 consensus_queue.cc:260] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.552173 24183 ts_tablet_manager.cc:1434] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:07:00.553748 24183 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.575965 24183 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.577032 24183 tablet_bootstrap.cc:492] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.577749 24183 ts_tablet_manager.cc:1403] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:07:00.580372 24183 raft_consensus.cc:359] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.581067 24183 raft_consensus.cc:740] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.581876 24183 consensus_queue.cc:260] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.584368 24183 ts_tablet_manager.cc:1434] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:00.585199 24183 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.599798 24183 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.600564 24183 tablet_bootstrap.cc:492] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.601207 24183 ts_tablet_manager.cc:1403] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:07:00.603418 24183 raft_consensus.cc:359] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.603938 24183 raft_consensus.cc:740] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.604571 24183 consensus_queue.cc:260] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.606369 24183 ts_tablet_manager.cc:1434] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:00.608013 24183 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.620792 24183 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.621608 24183 tablet_bootstrap.cc:492] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.622160 24183 ts_tablet_manager.cc:1403] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.007s
I20260709 13:07:00.624382 24183 raft_consensus.cc:359] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.624934 24183 raft_consensus.cc:740] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.625450 24183 consensus_queue.cc:260] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.627068 24183 ts_tablet_manager.cc:1434] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:00.628049 24183 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.642060 24183 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.643440 24183 tablet_bootstrap.cc:492] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.644186 24183 ts_tablet_manager.cc:1403] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:07:00.646680 24183 raft_consensus.cc:359] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.647424 24183 raft_consensus.cc:740] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.648056 24183 consensus_queue.cc:260] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.650056 24183 ts_tablet_manager.cc:1434] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:00.651430 24183 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.666251 24183 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.667490 24183 tablet_bootstrap.cc:492] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.668256 24183 ts_tablet_manager.cc:1403] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.017s	user 0.003s	sys 0.010s
I20260709 13:07:00.671125 24183 raft_consensus.cc:359] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.671940 24183 raft_consensus.cc:740] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.672748 24183 consensus_queue.cc:260] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.676574 24183 ts_tablet_manager.cc:1434] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.005s
I20260709 13:07:00.677542 24183 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.693996 24183 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.695207 24183 tablet_bootstrap.cc:492] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.695920 24183 ts_tablet_manager.cc:1403] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:07:00.698491 24183 raft_consensus.cc:359] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.699465 24183 raft_consensus.cc:740] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.700205 24183 consensus_queue.cc:260] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.701855 24183 ts_tablet_manager.cc:1434] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:00.702713 24183 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.717931 24183 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.718945 24183 tablet_bootstrap.cc:492] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.719633 24183 ts_tablet_manager.cc:1403] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:07:00.721731 24183 raft_consensus.cc:359] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.722218 24183 raft_consensus.cc:740] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.723095 24183 consensus_queue.cc:260] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.725103 24183 ts_tablet_manager.cc:1434] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:07:00.726217 24183 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.744498 24221 raft_consensus.cc:3060] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.772442 24184 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.773041 24184 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.776196 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" is_pre_election: true
I20260709 13:07:00.776646 24184 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:00.777024 24221 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 1.
I20260709 13:07:00.779058 23484 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:00.779836 24184 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:00.780558 24184 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.780956 24184 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:00.782169 24183 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.782495 24219 raft_consensus.cc:3060] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.783265 24183 tablet_bootstrap.cc:492] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.783996 24183 ts_tablet_manager.cc:1403] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.012s
I20260709 13:07:00.786587 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" is_pre_election: true
I20260709 13:07:00.787364 23444 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:07:00.786549 24183 raft_consensus.cc:359] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.788734 24184 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.789256 24183 raft_consensus.cc:740] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.790331 24183 consensus_queue.cc:260] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.791033 24184 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 election: Requested vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:00.792591 24183 ts_tablet_manager.cc:1434] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:00.793130 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:00.793185 24221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
I20260709 13:07:00.793845 23444 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:00.793992 24221 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:00.796320 24183 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.814981 24224 raft_consensus.cc:3060] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.815196 24221 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 3.
I20260709 13:07:00.816579 23484 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:00.818403 24283 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:00.818938 24283 raft_consensus.cc:697] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:00.819797 24283 consensus_queue.cc:237] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.836318 23444 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 3.
I20260709 13:07:00.869241 24183 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.870245 24183 tablet_bootstrap.cc:492] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.870960 24183 ts_tablet_manager.cc:1403] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.075s	user 0.042s	sys 0.018s
I20260709 13:07:00.873585 24183 raft_consensus.cc:359] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.874204 24183 raft_consensus.cc:740] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.876293 24183 consensus_queue.cc:260] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:00.878592 24183 ts_tablet_manager.cc:1434] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:00.879513 24183 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.896009 23266 catalog_manager.cc:5697] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 3, leader changed from d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:00.903292 24224 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.934676 24183 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.936015 24183 tablet_bootstrap.cc:492] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.936872 24183 ts_tablet_manager.cc:1403] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.009s
I20260709 13:07:00.939831 24183 raft_consensus.cc:359] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.940519 24183 raft_consensus.cc:740] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.941246 24183 consensus_queue.cc:260] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:00.943647 24183 ts_tablet_manager.cc:1434] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:00.944687 24183 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:00.988138 24183 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.989079 24183 tablet_bootstrap.cc:492] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:00.989706 24183 ts_tablet_manager.cc:1403] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.030s
I20260709 13:07:00.992190 24183 raft_consensus.cc:359] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.992873 24183 raft_consensus.cc:740] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.993546 24183 consensus_queue.cc:260] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:00.995613 24183 ts_tablet_manager.cc:1434] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:00.996423 24183 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap starting.
I20260709 13:07:01.017351 24183 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.018158 24183 tablet_bootstrap.cc:492] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Bootstrap complete.
I20260709 13:07:01.018745 24183 ts_tablet_manager.cc:1403] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:07:01.020676 24183 raft_consensus.cc:359] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:01.021418 24183 raft_consensus.cc:740] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.022095 24183 consensus_queue.cc:260] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:01.026074 24183 ts_tablet_manager.cc:1434] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:01.093008 24224 raft_consensus.cc:3060] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.183506 24224 raft_consensus.cc:3060] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.312325 23443 raft_consensus.cc:1275] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:01.313585 24283 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.326167 24221 raft_consensus.cc:1275] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:01.328660 24299 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:09.123251 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:09.175398 23515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44df4021944a47c9b442ffde13399f34"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:09.176184 23515 raft_consensus.cc:606] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.178889 23515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "546616724fd047fdb579028e4581131e"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:09.179342 23515 raft_consensus.cc:606] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.182185 23518 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "ceb56da8fd5145648a898f5487219c6e"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:09.182730 23518 raft_consensus.cc:606] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Received request to transfer leadership to TS ceb56da8fd5145648a898f5487219c6e
I20260709 13:07:09.186537 23518 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:09.187134 23518 raft_consensus.cc:606] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.198644 23440 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:09.199098 23440 raft_consensus.cc:606] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.201713 23518 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:09.202214 23518 raft_consensus.cc:606] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.214670 23590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:60858
I20260709 13:07:09.215211 23590 raft_consensus.cc:606] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:09.216383 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:07:09.217471 24387 raft_consensus.cc:993] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.218063 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:09.218091 24400 raft_consensus.cc:1081] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.220057 24224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44df4021944a47c9b442ffde13399f34"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52412
I20260709 13:07:09.220624 24224 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.221062 24224 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:09.228335 24224 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.230509 24224 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 4 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:09.241528 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:09.242165 23440 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:09.244793 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44df4021944a47c9b442ffde13399f34" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.245465 23518 raft_consensus.cc:3055] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:09.245657 23518 raft_consensus.cc:740] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:09.246534 23518 consensus_queue.cc:260] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.247607 23518 raft_consensus.cc:3060] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:09.251356 23440 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 4.
I20260709 13:07:09.252808 24177 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.253484 23518 raft_consensus.cc:2468] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 4.
I20260709 13:07:09.254802 24409 raft_consensus.cc:2804] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:09.255967 24409 raft_consensus.cc:697] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 4 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.257058 24409 consensus_queue.cc:237] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.263523 24400 raft_consensus.cc:993] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.265457 24387 raft_consensus.cc:1081] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.267623 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "546616724fd047fdb579028e4581131e"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52412
I20260709 13:07:09.268368 24221 raft_consensus.cc:493] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.268800 24221 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.278252 24221 raft_consensus.cc:515] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.279735 24221 leader_election.cc:290] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:09.279407 23267 catalog_manager.cc:5697] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 3 to 4, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 4 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.280575 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:09.281173 23440 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.282790 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "546616724fd047fdb579028e4581131e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.283428 23518 raft_consensus.cc:3055] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:09.283842 23518 raft_consensus.cc:740] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:09.285185 23518 consensus_queue.cc:260] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.286957 23518 raft_consensus.cc:3060] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.288164 23440 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:09.289651 24177 leader_election.cc:304] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.290558 24409 raft_consensus.cc:2804] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:09.291011 24409 raft_consensus.cc:697] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.291807 24409 consensus_queue.cc:237] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.295852 23518 raft_consensus.cc:2468] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:09.319579 23267 catalog_manager.cc:5697] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 2 to 3, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 3 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.376312 24413 raft_consensus.cc:993] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: : Instructing follower ceb56da8fd5145648a898f5487219c6e to start an election
I20260709 13:07:09.377023 24412 raft_consensus.cc:1081] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Signalling peer ceb56da8fd5145648a898f5487219c6e to start an election
I20260709 13:07:09.378754 23589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
 from {username='slave'} at 127.0.0.1:39758
I20260709 13:07:09.379213 23589 raft_consensus.cc:493] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.379585 23589 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.386756 23589 raft_consensus.cc:515] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:09.388731 23589 leader_election.cc:290] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:09.389611 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:09.389904 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d1599ca0cb14ed39d2ffde42649f5e6" candidate_uuid: "ceb56da8fd5145648a898f5487219c6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.390242 23440 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.390491 23518 raft_consensus.cc:3055] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:09.390836 23518 raft_consensus.cc:740] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:09.391532 23518 consensus_queue.cc:260] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:09.392475 23518 raft_consensus.cc:3060] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.394992 23440 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
I20260709 13:07:09.396356 23556 leader_election.cc:304] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:07:09.397246 24401 raft_consensus.cc:2804] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:09.397599 23518 raft_consensus.cc:2468] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb56da8fd5145648a898f5487219c6e in term 2.
I20260709 13:07:09.397651 24401 raft_consensus.cc:697] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Becoming Leader. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:09.398841 24401 consensus_queue.cc:237] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:09.407627 23267 catalog_manager.cc:5697] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e reported cstate change: term changed from 1 to 2, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to ceb56da8fd5145648a898f5487219c6e (127.13.163.196). New cstate: current_term: 2 leader_uuid: "ceb56da8fd5145648a898f5487219c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.648494 24400 raft_consensus.cc:993] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.649135 24413 raft_consensus.cc:1081] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.650715 24224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52412
I20260709 13:07:09.651170 24224 raft_consensus.cc:493] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.651491 24224 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.656579 24224 raft_consensus.cc:515] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.659173 24224 leader_election.cc:290] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:09.659415 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.660108 23515 raft_consensus.cc:3055] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:09.660434 23515 raft_consensus.cc:740] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:09.661273 23515 consensus_queue.cc:260] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.662568 23515 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.670953 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:09.671587 23589 raft_consensus.cc:3060] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.671557 23515 raft_consensus.cc:2468] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.673368 24177 leader_election.cc:304] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.674208 24409 raft_consensus.cc:2804] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:09.674680 24409 raft_consensus.cc:697] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.675570 24409 consensus_queue.cc:237] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.678706 23589 raft_consensus.cc:2468] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.687108 24413 raft_consensus.cc:993] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.687764 24413 raft_consensus.cc:1081] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.691442 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52412
I20260709 13:07:09.692080 24221 raft_consensus.cc:493] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.692914 24221 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.694223 23267 catalog_manager.cc:5697] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.701138 24221 raft_consensus.cc:515] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.705051 24221 leader_election.cc:290] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:09.706735 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.707588 23515 raft_consensus.cc:3055] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:09.708092 23515 raft_consensus.cc:740] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:09.709650 23515 consensus_queue.cc:260] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.715415 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1dd65a066e84285a82713f6ee51aa7e" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:09.716135 23589 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.716055 23515 raft_consensus.cc:3060] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.723088 23589 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.724337 24177 leader_election.cc:304] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.725291 24409 raft_consensus.cc:2804] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:09.725662 23515 raft_consensus.cc:2468] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.725930 24409 raft_consensus.cc:697] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.727610 24409 consensus_queue.cc:237] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:09.736373 24394 raft_consensus.cc:993] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.736827 24394 raft_consensus.cc:1081] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.738698 24222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52422
I20260709 13:07:09.739245 24222 raft_consensus.cc:493] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.739634 24222 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.739529 23267 catalog_manager.cc:5697] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.747573 24222 raft_consensus.cc:515] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.749547 24222 leader_election.cc:290] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:09.750499 23515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:09.750862 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e8e3d7d4343c88e56c92382808e9d" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:09.751185 23515 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.751473 23440 raft_consensus.cc:3055] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:09.751816 23440 raft_consensus.cc:740] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:09.752439 23440 consensus_queue.cc:260] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.753446 23440 raft_consensus.cc:3060] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.756373 23515 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:09.757489 24177 leader_election.cc:304] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.758276 24409 raft_consensus.cc:2804] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:09.758737 24409 raft_consensus.cc:697] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.759759 23440 raft_consensus.cc:2468] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:09.759616 24409 consensus_queue.cc:237] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:09.762657 24396 raft_consensus.cc:993] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.763427 24421 raft_consensus.cc:1081] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:09.765461 24222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52402
I20260709 13:07:09.766017 24222 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:09.766376 24222 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.768532 23515 raft_consensus.cc:1275] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:09.770601 24409 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:09.773556 23267 catalog_manager.cc:5697] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 2 to 3, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 3 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.775267 23518 raft_consensus.cc:1275] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:09.777040 24410 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:09.786007 24222 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:09.790287 23589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:09.790980 23589 raft_consensus.cc:3055] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:09.791267 23589 raft_consensus.cc:740] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:09.792045 23589 consensus_queue.cc:260] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:09.793279 23589 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.787757 24222 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:09.798491 23589 raft_consensus.cc:2468] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.801127 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f954a7fd7b3840a9b646d16a33cab480" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:09.801705 23440 raft_consensus.cc:3060] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:09.802428 24177 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:09.803341 24410 raft_consensus.cc:2804] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:09.815567 23441 raft_consensus.cc:1275] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:09.808732 23440 raft_consensus.cc:2468] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:09.817037 24421 consensus_queue.cc:1048] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:09.820245 24410 raft_consensus.cc:697] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:09.824321 24410 consensus_queue.cc:237] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:09.831099 23441 raft_consensus.cc:1275] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 4 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:09.833891 24427 consensus_queue.cc:1048] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:09.851234 23443 raft_consensus.cc:1275] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:09.853053 24429 consensus_queue.cc:1048] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:09.889470 23515 raft_consensus.cc:1275] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer ceb56da8fd5145648a898f5487219c6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:09.891052 24401 consensus_queue.cc:1048] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:09.973047 23268 catalog_manager.cc:5697] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from ceb56da8fd5145648a898f5487219c6e (127.13.163.196) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.080499 23518 raft_consensus.cc:1275] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.081844 24410 consensus_queue.cc:1048] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.094471 23587 raft_consensus.cc:1275] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.095965 24429 consensus_queue.cc:1048] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.159622 23441 raft_consensus.cc:1275] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:10.160931 24429 consensus_queue.cc:1048] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.168814 23587 raft_consensus.cc:1275] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.173759 23519 raft_consensus.cc:1275] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:10.176534 24410 consensus_queue.cc:1048] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.178679 24410 consensus_queue.cc:1048] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.202346 23515 raft_consensus.cc:1275] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.204102 24429 consensus_queue.cc:1048] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.219383 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:10.288635 23441 raft_consensus.cc:1275] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.289947 24429 consensus_queue.cc:1048] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.291100 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:10.305608 23587 raft_consensus.cc:1275] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:10.314684 24409 consensus_queue.cc:1048] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:10.345422 23441 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:10.346015 23441 raft_consensus.cc:606] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:10.349323 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:10.349895 23445 raft_consensus.cc:606] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:10.356181 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "12589c3399b641fc80f39058a475f8a1"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:10.356786 23445 raft_consensus.cc:606] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Received request to transfer leadership to TS 12589c3399b641fc80f39058a475f8a1
I20260709 13:07:10.361951 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:10.362530 23445 raft_consensus.cc:606] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:10.366887 23590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec789761c5344074974b9c23b79e418a"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:60858
I20260709 13:07:10.367483 23590 raft_consensus.cc:606] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:10.368651 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:10.370375 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:10.712549 24442 raft_consensus.cc:993] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.713294 24391 raft_consensus.cc:1081] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.714774 24222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52422
I20260709 13:07:10.715289 24222 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.715590 24222 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:10.722771 24222 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:10.724589 24222 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:10.725270 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:10.725672 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e2b1695b1849db90d73f0c6719c9d3" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:10.726073 23590 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:10.727099 23445 raft_consensus.cc:3055] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:10.727584 23445 raft_consensus.cc:740] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:10.728458 23445 consensus_queue.cc:260] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:10.729696 23445 raft_consensus.cc:3060] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:10.734896 23590 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:10.737579 24177 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:10.738438 24427 raft_consensus.cc:2804] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:10.738510 23445 raft_consensus.cc:2468] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 3.
I20260709 13:07:10.739013 24427 raft_consensus.cc:697] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:10.740028 24427 consensus_queue.cc:237] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:10.750680 23267 catalog_manager.cc:5697] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 2 to 3, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 3 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.798302 24391 raft_consensus.cc:993] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower 12589c3399b641fc80f39058a475f8a1 to start an election
I20260709 13:07:10.798823 24391 raft_consensus.cc:1081] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Signalling peer 12589c3399b641fc80f39058a475f8a1 to start an election
I20260709 13:07:10.799569 24394 raft_consensus.cc:993] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.800362 24442 raft_consensus.cc:1081] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.800521 23516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
 from {username='slave'} at 127.0.0.1:52180
I20260709 13:07:10.801215 23516 raft_consensus.cc:493] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.801604 23516 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.803581 24224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52422
I20260709 13:07:10.804347 24224 raft_consensus.cc:493] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.804760 24224 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.810158 23516 raft_consensus.cc:515] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:10.813330 24224 raft_consensus.cc:515] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.814523 23516 leader_election.cc:290] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:10.815532 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:10.816107 23590 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.816057 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:10.816728 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8678c528777f43c7b3552d5d8f484677" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:10.816901 23445 raft_consensus.cc:3055] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:10.817337 23441 raft_consensus.cc:3055] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:10.817778 23441 raft_consensus.cc:740] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:10.817888 24224 leader_election.cc:290] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:10.819022 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ece810899b40429ea55aa6a3642a63" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:10.818703 23441 consensus_queue.cc:260] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:10.817325 23445 raft_consensus.cc:740] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:10.819701 23516 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.820255 23441 raft_consensus.cc:3060] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.820166 23445 consensus_queue.cc:260] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.821552 23445 raft_consensus.cc:3060] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.823292 23590 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:07:10.824810 23482 leader_election.cc:304] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, ceb56da8fd5145648a898f5487219c6e; no voters: 
I20260709 13:07:10.825614 24413 raft_consensus.cc:2804] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:10.826716 24413 raft_consensus.cc:697] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Becoming Leader. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:10.827790 24413 consensus_queue.cc:237] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:10.835395 24442 raft_consensus.cc:993] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.836211 24442 raft_consensus.cc:1081] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.844763 23445 raft_consensus.cc:2468] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.846105 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52422
I20260709 13:07:10.846873 24221 raft_consensus.cc:493] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.847281 24221 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.845218 23516 raft_consensus.cc:2468] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.849107 24177 leader_election.cc:304] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:10.856104 23441 raft_consensus.cc:2468] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12589c3399b641fc80f39058a475f8a1 in term 2.
I20260709 13:07:10.856407 24409 raft_consensus.cc:2804] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:10.857326 24409 raft_consensus.cc:697] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:10.858568 24409 consensus_queue.cc:237] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.856137 24221 raft_consensus.cc:515] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.861635 24221 leader_election.cc:290] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247), 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741)
I20260709 13:07:10.864207 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:10.864900 23441 raft_consensus.cc:3055] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:10.865972 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d63cb871f04825b6880a4e77c0f97b" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:10.866627 23516 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.865330 23441 raft_consensus.cc:740] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:10.871438 23265 catalog_manager.cc:5697] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 reported cstate change: term changed from 1 to 2, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to 12589c3399b641fc80f39058a475f8a1 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "12589c3399b641fc80f39058a475f8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.872351 23441 consensus_queue.cc:260] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.874548 23441 raft_consensus.cc:3060] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.877326 23516 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.878611 24177 leader_election.cc:304] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:10.879545 24427 raft_consensus.cc:2804] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:10.880085 24427 raft_consensus.cc:697] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:10.881062 24427 consensus_queue.cc:237] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:10.886570 23441 raft_consensus.cc:2468] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.897200 23265 catalog_manager.cc:5697] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.916612 24421 raft_consensus.cc:993] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.917225 24421 raft_consensus.cc:1081] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:10.919430 24224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec789761c5344074974b9c23b79e418a"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52402
I20260709 13:07:10.920007 24224 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:10.920354 24224 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.919675 23265 catalog_manager.cc:5697] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:10.926968 24224 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:10.928922 24224 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:10.930083 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:10.930491 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec789761c5344074974b9c23b79e418a" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:10.931115 23518 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.930639 23590 raft_consensus.cc:3055] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:10.933049 23590 raft_consensus.cc:740] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:10.933709 23590 consensus_queue.cc:260] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:10.934651 23590 raft_consensus.cc:3060] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:10.936098 23518 raft_consensus.cc:2468] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.937208 24177 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:10.937916 24409 raft_consensus.cc:2804] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:10.938303 24409 raft_consensus.cc:697] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:10.939152 23590 raft_consensus.cc:2468] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:10.939050 24409 consensus_queue.cc:237] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:10.948101 23265 catalog_manager.cc:5697] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from ceb56da8fd5145648a898f5487219c6e (127.13.163.196) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:11.272437 23440 raft_consensus.cc:1275] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.273793 24409 consensus_queue.cc:1048] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.288954 23590 raft_consensus.cc:1275] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:11.290663 24409 consensus_queue.cc:1048] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:11.294044 23518 raft_consensus.cc:1275] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.297637 24429 consensus_queue.cc:1048] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:11.305738 23588 raft_consensus.cc:1275] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.310727 24413 consensus_queue.cc:1048] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.326678 23441 raft_consensus.cc:1275] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:11.329278 24429 consensus_queue.cc:1048] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:11.358933 23519 raft_consensus.cc:1275] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.363060 24429 consensus_queue.cc:1048] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.371272 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:11.394168 23516 raft_consensus.cc:1275] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.394495 23440 raft_consensus.cc:1275] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.399080 24429 consensus_queue.cc:1048] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.400913 24429 consensus_queue.cc:1048] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.405984 23441 raft_consensus.cc:1275] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer 12589c3399b641fc80f39058a475f8a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.407290 24413 consensus_queue.cc:1048] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.438265 23590 raft_consensus.cc:1275] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:11.440918 24409 consensus_queue.cc:1048] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:11.520008 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:11.556200 23518 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c"
dest_uuid: "12589c3399b641fc80f39058a475f8a1"
mode: GRACEFUL
new_leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
 from {username='slave'} at 127.0.0.1:57356
I20260709 13:07:11.556712 23518 raft_consensus.cc:606] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Received request to transfer leadership to TS 643f0d55f5ac4308a9f1d5e7158b43fe
I20260709 13:07:11.559350 23588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:60858
I20260709 13:07:11.559927 23588 raft_consensus.cc:606] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:11.562958 23588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:60858
I20260709 13:07:11.563478 23588 raft_consensus.cc:606] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:11.566166 23588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7d59103647944ec805367e074fecea6"
dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:60858
I20260709 13:07:11.566756 23588 raft_consensus.cc:606] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:11.567900 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:07:11.569450 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:11.673666 24413 raft_consensus.cc:993] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: : Instructing follower 643f0d55f5ac4308a9f1d5e7158b43fe to start an election
I20260709 13:07:11.674261 24465 raft_consensus.cc:1081] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Signalling peer 643f0d55f5ac4308a9f1d5e7158b43fe to start an election
I20260709 13:07:11.675756 23441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
 from {username='slave'} at 127.0.0.1:46156
I20260709 13:07:11.676328 23441 raft_consensus.cc:493] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:11.676671 23441 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.682415 23441 raft_consensus.cc:515] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:11.684314 23441 leader_election.cc:290] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:11.685223 23516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:11.685904 23516 raft_consensus.cc:3055] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:11.686008 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0664d2cef22d4b9fbcce815319e6e56c" candidate_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:11.686281 23516 raft_consensus.cc:740] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12589c3399b641fc80f39058a475f8a1, State: Running, Role: LEADER
I20260709 13:07:11.686724 23588 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.687255 23516 consensus_queue.cc:260] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:11.688678 23516 raft_consensus.cc:3060] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.694434 23588 raft_consensus.cc:2468] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:07:11.695593 23516 raft_consensus.cc:2468] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643f0d55f5ac4308a9f1d5e7158b43fe in term 2.
I20260709 13:07:11.698609 23408 leader_election.cc:304] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, 643f0d55f5ac4308a9f1d5e7158b43fe; no voters: 
I20260709 13:07:11.699453 24442 raft_consensus.cc:2804] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:11.699927 24442 raft_consensus.cc:697] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Becoming Leader. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:11.700609 24442 consensus_queue.cc:237] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } }
I20260709 13:07:11.712746 23267 catalog_manager.cc:5697] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe reported cstate change: term changed from 1 to 2, leader changed from 12589c3399b641fc80f39058a475f8a1 (127.13.163.195) to 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:11.843366 24477 raft_consensus.cc:993] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:11.843842 24401 raft_consensus.cc:1081] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:11.845346 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52402
I20260709 13:07:11.845791 24221 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:11.846076 24221 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.850351 24221 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.851759 24221 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:11.852689 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:11.853379 23518 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.853443 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3067bf3f594c4985b356c81a410f6c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:11.854126 23588 raft_consensus.cc:3055] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:11.854453 23588 raft_consensus.cc:740] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:11.855239 23588 consensus_queue.cc:260] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.856559 23588 raft_consensus.cc:3060] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.859510 23518 raft_consensus.cc:2468] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:11.860868 24177 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:11.861701 24409 raft_consensus.cc:2804] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:11.862373 23588 raft_consensus.cc:2468] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:11.862500 24409 raft_consensus.cc:697] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:11.864081 24409 consensus_queue.cc:237] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.871448 24477 raft_consensus.cc:993] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:11.871953 24477 raft_consensus.cc:1081] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:11.875576 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52402
I20260709 13:07:11.876081 24221 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:11.876416 24221 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.882414 24221 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.884282 24221 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791)
I20260709 13:07:11.886547 23518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12589c3399b641fc80f39058a475f8a1"
I20260709 13:07:11.887111 23518 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.887629 23267 catalog_manager.cc:5697] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from ceb56da8fd5145648a898f5487219c6e (127.13.163.196) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:11.892482 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0675ebf2d38f494d9b749b40e54bea4c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:11.893253 23588 raft_consensus.cc:3055] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:11.893571 23588 raft_consensus.cc:740] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:11.893821 23518 raft_consensus.cc:2468] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:11.894361 23588 consensus_queue.cc:260] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.895140 24177 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:11.895820 23588 raft_consensus.cc:3060] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.896338 24409 raft_consensus.cc:2804] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:11.899003 24409 raft_consensus.cc:697] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:11.900723 24409 consensus_queue.cc:237] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:11.905635 23588 raft_consensus.cc:2468] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:11.916921 23267 catalog_manager.cc:5697] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from ceb56da8fd5145648a898f5487219c6e (127.13.163.196) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:12.086687 24421 raft_consensus.cc:993] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:12.087364 24477 raft_consensus.cc:1081] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:12.090170 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7d59103647944ec805367e074fecea6"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52402
I20260709 13:07:12.091274 24221 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:12.091627 24221 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.098771 24221 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.101244 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:12.101867 23588 raft_consensus.cc:3055] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:12.102186 23588 raft_consensus.cc:740] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb56da8fd5145648a898f5487219c6e, State: Running, Role: LEADER
I20260709 13:07:12.102568 24221 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:12.103485 23440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d59103647944ec805367e074fecea6" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:12.104099 23440 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.105404 23588 consensus_queue.cc:260] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.108397 23588 raft_consensus.cc:3060] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.111255 23440 raft_consensus.cc:2468] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:12.112432 24177 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:12.113233 24409 raft_consensus.cc:2804] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:12.113705 24409 raft_consensus.cc:697] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:12.114554 24409 consensus_queue.cc:237] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.118114 23588 raft_consensus.cc:2468] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:12.126891 23267 catalog_manager.cc:5697] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from ceb56da8fd5145648a898f5487219c6e (127.13.163.196) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:12.277043 23516 raft_consensus.cc:1275] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.278381 24394 consensus_queue.cc:1048] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.299508 23590 raft_consensus.cc:1275] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer 643f0d55f5ac4308a9f1d5e7158b43fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.301661 24442 consensus_queue.cc:1048] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.396440 23590 raft_consensus.cc:1275] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.397611 24409 consensus_queue.cc:1048] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.421556 23516 raft_consensus.cc:1275] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.422928 24429 consensus_queue.cc:1048] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.472326 23516 raft_consensus.cc:1275] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.473658 24463 consensus_queue.cc:1048] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.478775 23590 raft_consensus.cc:1275] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.480443 24463 consensus_queue.cc:1048] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:12.552805 23440 raft_consensus.cc:1275] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.554327 24463 consensus_queue.cc:1048] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:12.566248 23590 raft_consensus.cc:1275] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:12.567924 24496 consensus_queue.cc:1048] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:12.570380 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:12.631174 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:12.676599 23441 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c"
dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
mode: GRACEFUL
new_leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:43464
I20260709 13:07:12.677323 23441 raft_consensus.cc:606] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Received request to transfer leadership to TS d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:12.678944 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:12.680750 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:12.723057 24442 raft_consensus.cc:993] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: : Instructing follower d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:12.723587 24394 raft_consensus.cc:1081] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Signalling peer d60ed2fc1ba64596942b6df1bf93b4ca to start an election
I20260709 13:07:12.724944 24221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c"
dest_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca"
 from {username='slave'} at 127.0.0.1:52422
I20260709 13:07:12.725404 24221 raft_consensus.cc:493] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:12.725662 24221 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.730854 24221 raft_consensus.cc:515] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.732219 24221 leader_election.cc:290] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Requested vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:12.733024 23590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb56da8fd5145648a898f5487219c6e"
I20260709 13:07:12.733734 23590 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.733786 23441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d9e995f5e94779b4b8d0a4f921a35c" candidate_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe"
I20260709 13:07:12.734390 23441 raft_consensus.cc:3055] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:12.734665 23441 raft_consensus.cc:740] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643f0d55f5ac4308a9f1d5e7158b43fe, State: Running, Role: LEADER
I20260709 13:07:12.735427 23441 consensus_queue.cc:260] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.736585 23441 raft_consensus.cc:3060] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:12.739715 23590 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:12.740926 24177 leader_election.cc:304] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca; no voters: 
I20260709 13:07:12.741699 24496 raft_consensus.cc:2804] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:12.742118 24496 raft_consensus.cc:697] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Becoming Leader. State: Replica: d60ed2fc1ba64596942b6df1bf93b4ca, State: Running, Role: LEADER
I20260709 13:07:12.742895 24496 consensus_queue.cc:237] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:12.743108 23441 raft_consensus.cc:2468] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ed2fc1ba64596942b6df1bf93b4ca in term 2.
I20260709 13:07:12.752452 23268 catalog_manager.cc:5697] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca reported cstate change: term changed from 1 to 2, leader changed from 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194) to d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193). New cstate: current_term: 2 leader_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.146339 23588 raft_consensus.cc:1275] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:13.147634 24496 consensus_queue.cc:1048] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:13.160310 23441 raft_consensus.cc:1275] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Refusing update from remote peer d60ed2fc1ba64596942b6df1bf93b4ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:13.161664 24496 consensus_queue.cc:1048] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:13.681654 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:13.722281 13967 tablet_server.cc:179] TabletServer@127.13.163.193:35013 shutting down...
W20260709 13:07:13.725016 23558 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 122 similar messages]
W20260709 13:07:13.728273 23558 consensus_peers.cc:597] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:13.746899 23484 consensus_peers.cc:597] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.769325 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:13.770648 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.771606 13967 raft_consensus.cc:2243] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.772573 13967 raft_consensus.cc:2272] T 0675ebf2d38f494d9b749b40e54bea4c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.775206 23558 consensus_peers.cc:597] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.776044 13967 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:13.776254 23410 consensus_peers.cc:597] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.777021 13967 raft_consensus.cc:2243] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.778039 13967 raft_consensus.cc:2272] T 6f3067bf3f594c4985b356c81a410f6c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.780983 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.781688 13967 raft_consensus.cc:2243] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.782253 13967 raft_consensus.cc:2272] T 5fa87a90cab0475da44d21f725d485ab P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.784688 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.785396 13967 raft_consensus.cc:2243] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.785928 13967 raft_consensus.cc:2272] T 0108a896781443dbaf9caef1e04e5cfa P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.788930 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.789578 13967 raft_consensus.cc:2243] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.789992 13967 raft_consensus.cc:2272] T 1c16a9c7d17249d98740f56a32c03d84 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.792986 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.793668 13967 raft_consensus.cc:2243] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.794474 13967 raft_consensus.cc:2272] T c20e8e3d7d4343c88e56c92382808e9d P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.796520 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.797139 13967 raft_consensus.cc:2243] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.797576 13967 raft_consensus.cc:2272] T b57c381a872f4c7aae7b11e12cc439b8 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.800338 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.801215 13967 raft_consensus.cc:2243] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.801674 13967 raft_consensus.cc:2272] T 0ce2a928268b4d73afa32a47bbbcbd64 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.801851 23410 consensus_peers.cc:597] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.803876 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.804584 13967 raft_consensus.cc:2243] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.804997 13967 raft_consensus.cc:2272] T 5fa7940429cf448f96c400f792bb6580 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.807202 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.807818 13967 raft_consensus.cc:2243] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.808308 13967 raft_consensus.cc:2272] T 538d207f493d4b14b97bce7658551672 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.810447 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.811017 13967 raft_consensus.cc:2243] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.811430 13967 raft_consensus.cc:2272] T 3a3cf23d64f04f4cb98bbd43988743ea P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.813568 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.814103 13967 raft_consensus.cc:2243] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.814523 13967 raft_consensus.cc:2272] T 31308be47968488d8fe1db62fc5e9d48 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.816704 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.817327 13967 raft_consensus.cc:2243] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.818284 13967 raft_consensus.cc:2272] T 9af1e469bb524d288ab7911185a018b2 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.820614 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.821336 13967 raft_consensus.cc:2243] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.821868 13967 raft_consensus.cc:2272] T be3f330957084b66847db5f71ee7bc36 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.822203 23410 consensus_peers.cc:597] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.824474 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.825217 13967 raft_consensus.cc:2243] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.826128 13967 raft_consensus.cc:2272] T e1e2b1695b1849db90d73f0c6719c9d3 P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.826959 23484 consensus_peers.cc:597] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.828204 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.828949 13967 raft_consensus.cc:2243] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.829654 13967 raft_consensus.cc:2272] T 44df4021944a47c9b442ffde13399f34 P d60ed2fc1ba64596942b6df1bf93b4ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.831840 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.832382 13967 raft_consensus.cc:2243] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.833217 13967 raft_consensus.cc:2272] T 02d63cb871f04825b6880a4e77c0f97b P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.835469 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.836128 13967 raft_consensus.cc:2243] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.836575 13967 raft_consensus.cc:2272] T 8118adb799df4bc2adcd22faba9e44d6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.838706 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.839282 13967 raft_consensus.cc:2243] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.840176 13967 raft_consensus.cc:2272] T ec789761c5344074974b9c23b79e418a P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.842401 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.842996 13967 raft_consensus.cc:2243] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.843454 13967 raft_consensus.cc:2272] T c4bb4f04978d43798dc3c00c13d510be P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.845729 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.846285 13967 raft_consensus.cc:2243] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.846689 13967 raft_consensus.cc:2272] T 787a4af7968d4c359c9efbc9abfe9b40 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.848899 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.849436 13967 raft_consensus.cc:2243] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.849843 13967 raft_consensus.cc:2272] T 64b73b48f1244e19a12b43c7f7a82cc5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.851914 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.852463 13967 raft_consensus.cc:2243] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.853278 13967 raft_consensus.cc:2272] T 37ece810899b40429ea55aa6a3642a63 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.855124 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.855679 13967 raft_consensus.cc:2243] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.856062 13967 raft_consensus.cc:2272] T a7ef8cd1ea7446f6bd4a4f133cbf701c P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.858009 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.858705 13967 raft_consensus.cc:2243] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.859125 13967 raft_consensus.cc:2272] T 0ddf62d6f4e045f8b06e7170496fda73 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.861745 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.862303 13967 raft_consensus.cc:2243] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.862699 13967 raft_consensus.cc:2272] T d1fb77caf9eb4699b2319185288758e0 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.865746 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.866484 13967 raft_consensus.cc:2243] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.866900 13967 raft_consensus.cc:2272] T c366e593434e49a9b0f8d3a41616077d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.867113 23410 consensus_peers.cc:597] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.869544 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.870177 13967 raft_consensus.cc:2243] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.870954 13967 raft_consensus.cc:2272] T 70d9e995f5e94779b4b8d0a4f921a35c P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.873625 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.874387 13967 raft_consensus.cc:2243] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.874892 13967 raft_consensus.cc:2272] T ff658c154728474ab62925bcfa9e4fae P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.876775 23558 consensus_peers.cc:597] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.877493 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.878180 13967 raft_consensus.cc:2243] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.878649 13967 raft_consensus.cc:2272] T 3a01f067a6214e979fafb367a095f1c1 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.881588 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.882341 13967 raft_consensus.cc:2243] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.883116 13967 raft_consensus.cc:2272] T d1dd65a066e84285a82713f6ee51aa7e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.885840 13967 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:13.886325 23558 consensus_peers.cc:597] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.886587 13967 raft_consensus.cc:2243] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.887089 13967 raft_consensus.cc:2272] T ae7dba7096144600b277ae19d383145e P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.890018 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.890838 13967 raft_consensus.cc:2243] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.891279 13967 raft_consensus.cc:2272] T 671e4639fc5945aba1d87c67ed1fc249 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.894243 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.894943 13967 raft_consensus.cc:2243] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.895514 13967 raft_consensus.cc:2272] T aa6c9e6d5c9540229369e18bc85a6a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.897902 23484 consensus_peers.cc:597] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.898013 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.898656 13967 raft_consensus.cc:2243] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.899077 13967 raft_consensus.cc:2272] T 35d1d7ecd2e941d289aefe4228c0e673 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.901093 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.901656 13967 raft_consensus.cc:2243] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.902366 13967 raft_consensus.cc:2272] T f7d59103647944ec805367e074fecea6 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.904448 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.905186 13967 raft_consensus.cc:2243] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:13.905929 13967 raft_consensus.cc:2272] T f954a7fd7b3840a9b646d16a33cab480 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.908581 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.909327 13967 raft_consensus.cc:2243] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:07:13.910264 23558 consensus_peers.cc:597] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.910271 13967 raft_consensus.cc:2272] T 546616724fd047fdb579028e4581131e P d60ed2fc1ba64596942b6df1bf93b4ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.912819 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.913612 13967 raft_consensus.cc:2243] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.914083 13967 raft_consensus.cc:2272] T 893c1c5680b947488f9d810e532fe7e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.914448 23484 consensus_peers.cc:597] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:13.916829 23558 consensus_peers.cc:597] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.917060 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.917929 13967 raft_consensus.cc:2243] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.918298 13967 raft_consensus.cc:2272] T 376a2904d9ff484195a63051a29b8a0d P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.920423 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.921121 13967 raft_consensus.cc:2243] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.921584 13967 raft_consensus.cc:2272] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.924268 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.924950 13967 raft_consensus.cc:2243] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.925484 13967 raft_consensus.cc:2272] T c8912a0f40244d6bb5ff749263ff4993 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.928074 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.928722 13967 raft_consensus.cc:2243] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.929244 13967 raft_consensus.cc:2272] T 18748afbb84547cba3ae58d7476a2119 P d60ed2fc1ba64596942b6df1bf93b4ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.931942 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.932564 13967 raft_consensus.cc:2243] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.932996 13967 raft_consensus.cc:2272] T 31f1ce58a33542c395d1e4e41fe71ea7 P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:13.935526 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:13.935933 13967 raft_consensus.cc:2243] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:13.936275 13967 raft_consensus.cc:2272] T ce891b8160664205892014cec10858ff P d60ed2fc1ba64596942b6df1bf93b4ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:13.944737 23410 consensus_peers.cc:597] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:13.992403 23410 consensus_peers.cc:597] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:13.998975 13967 tablet_server.cc:196] TabletServer@127.13.163.193:35013 shutdown complete.
W20260709 13:07:14.002712 23558 consensus_peers.cc:597] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.041617 23410 consensus_peers.cc:597] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.043666 23558 consensus_peers.cc:597] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.044478 23410 consensus_peers.cc:597] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.048622 23484 consensus_peers.cc:597] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.097671 23558 consensus_peers.cc:597] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.100454 23484 consensus_peers.cc:597] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.102440 23558 consensus_peers.cc:597] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.107982 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
W20260709 13:07:14.133422 23410 consensus_peers.cc:597] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.134280 23482 consensus_peers.cc:597] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.145673 23558 consensus_peers.cc:597] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.154138 23558 consensus_peers.cc:597] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.154685 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:14.155519 13967 tablet_replica.cc:333] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.156363 13967 raft_consensus.cc:2243] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.157414 13967 raft_consensus.cc:2272] T 64b73b48f1244e19a12b43c7f7a82cc5 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.160305 13967 tablet_replica.cc:333] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.161139 13967 raft_consensus.cc:2243] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.161567 13967 raft_consensus.cc:2272] T 37d36e08707f4d8295154d67b1450b0f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.164294 13967 tablet_replica.cc:333] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.165091 13967 raft_consensus.cc:2243] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.165607 13967 raft_consensus.cc:2272] T 3038aa8132294ba7832fad08934a848f P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.167084 23556 consensus_peers.cc:597] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.168478 13967 tablet_replica.cc:333] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.169452 13967 raft_consensus.cc:2243] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.169932 13967 raft_consensus.cc:2272] T c20e8e3d7d4343c88e56c92382808e9d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.172137 13967 tablet_replica.cc:333] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.173017 13967 raft_consensus.cc:2243] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.174088 13967 raft_consensus.cc:2272] T 4743addeb1314388b9469d11e0ff9385 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.176206 13967 tablet_replica.cc:333] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.176975 13967 raft_consensus.cc:2243] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.177372 13967 raft_consensus.cc:2272] T b57c381a872f4c7aae7b11e12cc439b8 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.179214 13967 tablet_replica.cc:333] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.179840 13967 raft_consensus.cc:2243] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.180251 13967 raft_consensus.cc:2272] T ce891b8160664205892014cec10858ff P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.182163 13967 tablet_replica.cc:333] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.182693 13967 raft_consensus.cc:2243] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.183100 13967 raft_consensus.cc:2272] T 31f1ce58a33542c395d1e4e41fe71ea7 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.185137 13967 tablet_replica.cc:333] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.185712 13967 raft_consensus.cc:2243] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.186486 13967 raft_consensus.cc:2272] T 18748afbb84547cba3ae58d7476a2119 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.188509 13967 tablet_replica.cc:333] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.189141 13967 raft_consensus.cc:2243] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.189855 13967 raft_consensus.cc:2272] T 0664d2cef22d4b9fbcce815319e6e56c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.191754 13967 tablet_replica.cc:333] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.192260 13967 raft_consensus.cc:2243] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.192677 13967 raft_consensus.cc:2272] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.194895 13967 tablet_replica.cc:333] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.195582 13967 raft_consensus.cc:2243] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.196089 13967 raft_consensus.cc:2272] T 0ce2a928268b4d73afa32a47bbbcbd64 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.198577 13967 tablet_replica.cc:333] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.199106 13967 raft_consensus.cc:2243] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.199853 13967 raft_consensus.cc:2272] T 5fa87a90cab0475da44d21f725d485ab P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.201772 13967 tablet_replica.cc:333] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.202297 13967 raft_consensus.cc:2243] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.202720 13967 raft_consensus.cc:2272] T 0108a896781443dbaf9caef1e04e5cfa P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.204753 13967 tablet_replica.cc:333] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.205331 13967 raft_consensus.cc:2243] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.205765 13967 raft_consensus.cc:2272] T 56c0cd2ce17e48709b4a82c121539b62 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.207589 13967 tablet_replica.cc:333] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.208135 13967 raft_consensus.cc:2243] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.208536 13967 raft_consensus.cc:2272] T 44df4021944a47c9b442ffde13399f34 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.210484 13967 tablet_replica.cc:333] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.210947 13967 raft_consensus.cc:2243] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.211279 13967 raft_consensus.cc:2272] T e1e2b1695b1849db90d73f0c6719c9d3 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.213065 13967 tablet_replica.cc:333] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.213532 13967 raft_consensus.cc:2243] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.213871 13967 raft_consensus.cc:2272] T 02d63cb871f04825b6880a4e77c0f97b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.215592 13967 tablet_replica.cc:333] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.216086 13967 raft_consensus.cc:2243] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.216820 13967 raft_consensus.cc:2272] T 5fa7940429cf448f96c400f792bb6580 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.219074 13967 tablet_replica.cc:333] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.219792 13967 raft_consensus.cc:2243] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.220294 13967 raft_consensus.cc:2272] T fc4a932d15eb4e9e89e5b9bf6627de2b P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.222069 23556 consensus_peers.cc:597] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.223021 13967 tablet_replica.cc:333] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.223845 13967 raft_consensus.cc:2243] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.224742 13967 raft_consensus.cc:2272] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.225095 23482 consensus_peers.cc:597] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.227116 13967 tablet_replica.cc:333] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.227926 13967 raft_consensus.cc:2243] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.229007 13967 raft_consensus.cc:2272] T 8118adb799df4bc2adcd22faba9e44d6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.231032 13967 tablet_replica.cc:333] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.231721 13967 raft_consensus.cc:2243] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.232204 13967 raft_consensus.cc:2272] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.234596 13967 tablet_replica.cc:333] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.235340 13967 raft_consensus.cc:2243] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.235896 13967 raft_consensus.cc:2272] T aa6c9e6d5c9540229369e18bc85a6a0d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.236817 23556 consensus_peers.cc:597] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.238358 13967 tablet_replica.cc:333] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.238988 13967 raft_consensus.cc:2243] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.239758 13967 raft_consensus.cc:2272] T 31308be47968488d8fe1db62fc5e9d48 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.241997 13967 tablet_replica.cc:333] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.242571 13967 raft_consensus.cc:2243] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.243405 13967 raft_consensus.cc:2272] T ae7dba7096144600b277ae19d383145e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.245955 13967 tablet_replica.cc:333] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.246587 13967 raft_consensus.cc:2243] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.247478 13967 raft_consensus.cc:2272] T 671e4639fc5945aba1d87c67ed1fc249 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.249902 13967 tablet_replica.cc:333] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.250555 13967 raft_consensus.cc:2243] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.251436 13967 raft_consensus.cc:2272] T 3a01f067a6214e979fafb367a095f1c1 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.253994 13967 tablet_replica.cc:333] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.254807 13967 raft_consensus.cc:2243] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.255434 13967 raft_consensus.cc:2272] T c366e593434e49a9b0f8d3a41616077d P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.258316 13967 tablet_replica.cc:333] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.258963 13967 raft_consensus.cc:2243] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.259509 13967 raft_consensus.cc:2272] T 21203c7d4b1148e1a4e659daf1df428c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.261637 13967 tablet_replica.cc:333] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.262220 13967 raft_consensus.cc:2243] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.262637 13967 raft_consensus.cc:2272] T 70d9e995f5e94779b4b8d0a4f921a35c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.264647 13967 tablet_replica.cc:333] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.265301 13967 raft_consensus.cc:2243] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.265785 13967 raft_consensus.cc:2272] T c8912a0f40244d6bb5ff749263ff4993 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.267892 13967 tablet_replica.cc:333] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.268613 13967 raft_consensus.cc:2243] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.269164 13967 raft_consensus.cc:2272] T f7d59103647944ec805367e074fecea6 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.272112 13967 tablet_replica.cc:333] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.272883 13967 raft_consensus.cc:2243] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.273423 13967 raft_consensus.cc:2272] T f954a7fd7b3840a9b646d16a33cab480 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.276114 13967 tablet_replica.cc:333] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.279713 13967 raft_consensus.cc:2243] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.280280 13967 raft_consensus.cc:2272] T 546616724fd047fdb579028e4581131e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.283097 13967 tablet_replica.cc:333] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.284009 13967 raft_consensus.cc:2243] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.285058 13967 raft_consensus.cc:2272] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.287786 13967 tablet_replica.cc:333] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.288487 13967 raft_consensus.cc:2243] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.289008 13967 raft_consensus.cc:2272] T 37ece810899b40429ea55aa6a3642a63 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.291631 13967 tablet_replica.cc:333] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.292373 13967 raft_consensus.cc:2243] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.292935 13967 raft_consensus.cc:2272] T 8678c528777f43c7b3552d5d8f484677 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.293421 23556 consensus_peers.cc:597] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.295608 13967 tablet_replica.cc:333] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.296429 13967 raft_consensus.cc:2243] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.297013 13967 raft_consensus.cc:2272] T c4bb4f04978d43798dc3c00c13d510be P 643f0d55f5ac4308a9f1d5e7158b43fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.324935 13967 tablet_replica.cc:333] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.335866 13967 raft_consensus.cc:2243] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.339461 13967 raft_consensus.cc:2272] T 5e66e5d5789d4dc3a56f76f34b603bad P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.358444 13967 tablet_replica.cc:333] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.360145 13967 raft_consensus.cc:2243] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.361563 13967 raft_consensus.cc:2272] T cc86fab625684e049569b12b52fdac33 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.358060 23558 consensus_peers.cc:597] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.367574 13967 tablet_replica.cc:333] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.369410 13967 raft_consensus.cc:2243] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.370837 13967 raft_consensus.cc:2272] T 77c7e59145ae42669dbe05a9943aa20e P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.377521 13967 tablet_replica.cc:333] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.378505 13967 raft_consensus.cc:2243] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.379597 13967 raft_consensus.cc:2272] T 0ddf62d6f4e045f8b06e7170496fda73 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.385509 13967 tablet_replica.cc:333] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe: stopping tablet replica
I20260709 13:07:14.386438 13967 raft_consensus.cc:2243] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.389477 13967 raft_consensus.cc:2272] T d1fb77caf9eb4699b2319185288758e0 P 643f0d55f5ac4308a9f1d5e7158b43fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.396965 23556 consensus_peers.cc:597] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.409301 23484 consensus_peers.cc:597] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 -> Peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Couldn't send request to peer d60ed2fc1ba64596942b6df1bf93b4ca. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.426050 23556 consensus_peers.cc:597] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.428903 23482 consensus_peers.cc:597] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.429790 23482 consensus_peers.cc:597] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.447860 23556 consensus_peers.cc:597] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.494280 23482 consensus_peers.cc:597] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.501586 23482 consensus_peers.cc:597] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.521189 23482 consensus_peers.cc:597] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.544487 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
W20260709 13:07:14.563830 23556 consensus_peers.cc:597] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.569607 23482 consensus_peers.cc:597] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.572150 23556 consensus_peers.cc:597] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.594785 23482 consensus_peers.cc:597] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.621426 23482 consensus_peers.cc:597] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:14.651669 23556 consensus_peers.cc:597] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e -> Peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Couldn't send request to peer 643f0d55f5ac4308a9f1d5e7158b43fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.665900 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
W20260709 13:07:14.674861 23556 consensus_peers.cc:597] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.700640 24413 raft_consensus.cc:493] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.701293 24413 raft_consensus.cc:515] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:14.704625 23588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af1e469bb524d288ab7911185a018b2" candidate_uuid: "12589c3399b641fc80f39058a475f8a1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ceb56da8fd5145648a898f5487219c6e" is_pre_election: true
W20260709 13:07:14.715045 23556 consensus_peers.cc:597] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.717084 24413 leader_election.cc:290] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ceb56da8fd5145648a898f5487219c6e (127.13.163.196:41791), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:07:14.718745 23484 leader_election.cc:336] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:14.719286 23484 leader_election.cc:304] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: ceb56da8fd5145648a898f5487219c6e, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.720341 24400 raft_consensus.cc:2749] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.727439 24400 raft_consensus.cc:493] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.728257 24400 raft_consensus.cc:515] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } } peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } }
I20260709 13:07:14.730608 24400 leader_election.cc:290] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:14.731094 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:14.732170 13967 tablet_replica.cc:333] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.733388 13967 raft_consensus.cc:2243] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.734104 13967 raft_consensus.cc:2272] T 787a4af7968d4c359c9efbc9abfe9b40 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.735158 23484 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:07:14.735847 23482 leader_election.cc:336] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111)
I20260709 13:07:14.736411 23482 leader_election.cc:304] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12589c3399b641fc80f39058a475f8a1; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.737133 24400 raft_consensus.cc:2749] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.738338 13967 tablet_replica.cc:333] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.739652 13967 raft_consensus.cc:2243] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.740484 13967 raft_consensus.cc:2272] T 64b73b48f1244e19a12b43c7f7a82cc5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.743497 13967 tablet_replica.cc:333] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.744244 13967 raft_consensus.cc:2243] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.745338 13967 raft_consensus.cc:2272] T 3038aa8132294ba7832fad08934a848f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.747851 13967 tablet_replica.cc:333] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.748623 13967 raft_consensus.cc:2243] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.749322 13967 raft_consensus.cc:2272] T 4743addeb1314388b9469d11e0ff9385 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.752667 13967 tablet_replica.cc:333] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.753681 13967 raft_consensus.cc:2243] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.754468 13967 raft_consensus.cc:2272] T ec789761c5344074974b9c23b79e418a P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.757619 13967 tablet_replica.cc:333] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.758279 13967 raft_consensus.cc:2243] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.758971 13967 raft_consensus.cc:2272] T 4d8c839e30fa42ee9ff04b9ba759fc85 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.763000 13967 tablet_replica.cc:333] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.764151 13967 raft_consensus.cc:2243] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.764991 13967 raft_consensus.cc:2272] T 376a2904d9ff484195a63051a29b8a0d P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.767983 13967 tablet_replica.cc:333] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.768778 13967 raft_consensus.cc:2243] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.769483 13967 raft_consensus.cc:2272] T 6f3067bf3f594c4985b356c81a410f6c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.772126 13967 tablet_replica.cc:333] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.773046 13967 raft_consensus.cc:2243] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.774209 13967 raft_consensus.cc:2272] T 90f7c7f831d14ec5870cb93e9b1f9f02 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.777240 13967 tablet_replica.cc:333] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.778188 13967 raft_consensus.cc:2243] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.778970 13967 raft_consensus.cc:2272] T a7ef8cd1ea7446f6bd4a4f133cbf701c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.782284 13967 tablet_replica.cc:333] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.783007 13967 raft_consensus.cc:2243] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.783607 13967 raft_consensus.cc:2272] T 37ece810899b40429ea55aa6a3642a63 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.786127 13967 tablet_replica.cc:333] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.786840 13967 raft_consensus.cc:2243] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.787411 13967 raft_consensus.cc:2272] T 18748afbb84547cba3ae58d7476a2119 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.789350 24519 raft_consensus.cc:493] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.790548 13967 tablet_replica.cc:333] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.790089 24519 raft_consensus.cc:515] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:14.791630 13967 raft_consensus.cc:2243] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.792495 13967 raft_consensus.cc:2272] T 0664d2cef22d4b9fbcce815319e6e56c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.793040 24519 leader_election.cc:290] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
I20260709 13:07:14.796520 13967 tablet_replica.cc:333] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.797842 13967 raft_consensus.cc:2243] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:14.796600 23556 leader_election.cc:336] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:14.798733 23558 leader_election.cc:336] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:14.798889 13967 raft_consensus.cc:2272] T ae7dba7096144600b277ae19d383145e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.799333 23558 leader_election.cc:304] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.800302 24519 raft_consensus.cc:2749] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.802407 13967 tablet_replica.cc:333] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.803189 13967 raft_consensus.cc:2243] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.803813 13967 raft_consensus.cc:2272] T d1dd65a066e84285a82713f6ee51aa7e P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.806384 13967 tablet_replica.cc:333] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.807128 13967 raft_consensus.cc:2243] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.807675 13967 raft_consensus.cc:2272] T 671e4639fc5945aba1d87c67ed1fc249 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.810349 13967 tablet_replica.cc:333] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.811029 13967 raft_consensus.cc:2243] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.811975 13967 raft_consensus.cc:2272] T ff658c154728474ab62925bcfa9e4fae P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.814605 13967 tablet_replica.cc:333] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.815295 13967 raft_consensus.cc:2243] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.815889 13967 raft_consensus.cc:2272] T 3a01f067a6214e979fafb367a095f1c1 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.818792 13967 tablet_replica.cc:333] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.819595 13967 raft_consensus.cc:2243] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.821130 13967 raft_consensus.cc:2272] T 37d36e08707f4d8295154d67b1450b0f P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.824743 13967 tablet_replica.cc:333] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.825798 13967 raft_consensus.cc:2243] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.826900 13967 raft_consensus.cc:2272] T 3a3cf23d64f04f4cb98bbd43988743ea P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.829389 13967 tablet_replica.cc:333] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.829977 13967 raft_consensus.cc:2243] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.830971 13967 raft_consensus.cc:2272] T 538d207f493d4b14b97bce7658551672 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.833913 13967 tablet_replica.cc:333] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.834904 13967 raft_consensus.cc:2243] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.835517 13967 raft_consensus.cc:2272] T 56c0cd2ce17e48709b4a82c121539b62 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.838986 13967 tablet_replica.cc:333] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.839836 13967 raft_consensus.cc:2243] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.840437 13967 raft_consensus.cc:2272] T 44df4021944a47c9b442ffde13399f34 P 12589c3399b641fc80f39058a475f8a1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.843128 13967 tablet_replica.cc:333] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.844017 13967 raft_consensus.cc:2243] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.844703 13967 raft_consensus.cc:2272] T be3f330957084b66847db5f71ee7bc36 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.847340 13967 tablet_replica.cc:333] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.848007 13967 raft_consensus.cc:2243] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.848554 13967 raft_consensus.cc:2272] T 02d63cb871f04825b6880a4e77c0f97b P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.851058 13967 tablet_replica.cc:333] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.851778 13967 raft_consensus.cc:2243] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.852314 13967 raft_consensus.cc:2272] T 5fa7940429cf448f96c400f792bb6580 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.855156 13967 tablet_replica.cc:333] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.855844 13967 raft_consensus.cc:2243] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.856906 13967 raft_consensus.cc:2272] T fc4a932d15eb4e9e89e5b9bf6627de2b P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.859357 13967 tablet_replica.cc:333] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.860009 13967 raft_consensus.cc:2243] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.860561 13967 raft_consensus.cc:2272] T 9af1e469bb524d288ab7911185a018b2 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.863078 13967 tablet_replica.cc:333] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.863687 13967 raft_consensus.cc:2243] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.864236 13967 raft_consensus.cc:2272] T 31308be47968488d8fe1db62fc5e9d48 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.867238 13967 tablet_replica.cc:333] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.868157 13967 raft_consensus.cc:2243] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.869458 13967 raft_consensus.cc:2272] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.872752 13967 tablet_replica.cc:333] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.873788 13967 raft_consensus.cc:2243] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.875123 13967 raft_consensus.cc:2272] T c8912a0f40244d6bb5ff749263ff4993 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.877710 13967 tablet_replica.cc:333] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.878309 13967 raft_consensus.cc:2243] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.879355 13967 raft_consensus.cc:2272] T 21203c7d4b1148e1a4e659daf1df428c P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.882071 13967 tablet_replica.cc:333] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.882791 13967 raft_consensus.cc:2243] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.883780 13967 raft_consensus.cc:2272] T 8678c528777f43c7b3552d5d8f484677 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.886310 13967 tablet_replica.cc:333] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.886984 13967 raft_consensus.cc:2243] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.887615 13967 raft_consensus.cc:2272] T 0675ebf2d38f494d9b749b40e54bea4c P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.890360 13967 tablet_replica.cc:333] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.890969 13967 raft_consensus.cc:2243] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.891577 13967 raft_consensus.cc:2272] T 7d1599ca0cb14ed39d2ffde42649f5e6 P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.894171 13967 tablet_replica.cc:333] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.894861 13967 raft_consensus.cc:2243] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.895521 13967 raft_consensus.cc:2272] T 35d1d7ecd2e941d289aefe4228c0e673 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.898967 13967 tablet_replica.cc:333] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.899616 13967 raft_consensus.cc:2243] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.900661 13967 raft_consensus.cc:2272] T aa6c9e6d5c9540229369e18bc85a6a0d P 12589c3399b641fc80f39058a475f8a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.903084 13967 tablet_replica.cc:333] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.903661 13967 raft_consensus.cc:2243] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.904124 13967 raft_consensus.cc:2272] T 546616724fd047fdb579028e4581131e P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.906730 13967 tablet_replica.cc:333] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.907243 13967 raft_consensus.cc:2243] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.907646 13967 raft_consensus.cc:2272] T 893c1c5680b947488f9d810e532fe7e5 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.910190 13967 tablet_replica.cc:333] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.910744 13967 raft_consensus.cc:2243] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.911259 13967 raft_consensus.cc:2272] T 5e66e5d5789d4dc3a56f76f34b603bad P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.913699 13967 tablet_replica.cc:333] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.914346 13967 raft_consensus.cc:2243] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.915256 13967 raft_consensus.cc:2272] T cc86fab625684e049569b12b52fdac33 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.917644 13967 tablet_replica.cc:333] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.918139 13967 raft_consensus.cc:2243] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:14.918699 13967 raft_consensus.cc:2272] T c20e8e3d7d4343c88e56c92382808e9d P 12589c3399b641fc80f39058a475f8a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.921171 13967 tablet_replica.cc:333] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.921701 13967 raft_consensus.cc:2243] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.922544 13967 raft_consensus.cc:2272] T 1c16a9c7d17249d98740f56a32c03d84 P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:14.925045 13967 tablet_replica.cc:333] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1: stopping tablet replica
I20260709 13:07:14.925720 13967 raft_consensus.cc:2243] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:14.926698 13967 raft_consensus.cc:2272] T 77c7e59145ae42669dbe05a9943aa20e P 12589c3399b641fc80f39058a475f8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:14.928279 23556 consensus_peers.cc:597] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.937543 24519 raft_consensus.cc:493] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.938153 24519 raft_consensus.cc:515] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
W20260709 13:07:14.939652 23556 consensus_peers.cc:597] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.940469 24519 leader_election.cc:290] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:07:14.943879 23556 leader_election.cc:336] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:14.947933 23558 leader_election.cc:336] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:14.948381 23558 leader_election.cc:304] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.949522 24519 raft_consensus.cc:2749] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:14.964996 23556 consensus_peers.cc:597] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.978479 24519 raft_consensus.cc:493] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.979106 24519 raft_consensus.cc:515] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:14.981153 24519 leader_election.cc:290] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:07:14.985162 23558 leader_election.cc:336] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:07:14.986536 23556 leader_election.cc:336] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111)
I20260709 13:07:14.986914 23556 leader_election.cc:304] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.987543 24519 raft_consensus.cc:2749] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.988109 24508 raft_consensus.cc:493] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:14.988770 24508 raft_consensus.cc:515] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:14.990752 24508 leader_election.cc:290] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:07:14.994064 23556 leader_election.cc:336] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:14.995818 23558 leader_election.cc:336] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:14.996250 23558 leader_election.cc:304] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:14.997496 24519 raft_consensus.cc:2749] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:14.999010 23556 consensus_peers.cc:597] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:15.035356 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:07:15.109158 13967 tablet_server.cc:179] TabletServer@127.13.163.196:0 shutting down...
I20260709 13:07:15.116153 24519 raft_consensus.cc:493] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:15.116888 24519 raft_consensus.cc:515] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12589c3399b641fc80f39058a475f8a1" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36741 } } peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } }
I20260709 13:07:15.119271 24519 leader_election.cc:290] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741), d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013)
W20260709 13:07:15.126322 23556 leader_election.cc:336] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Network error: Client connection negotiation failed: client connection to 127.13.163.195:36741: connect: Connection refused (error 111)
W20260709 13:07:15.127707 23558 leader_election.cc:336] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:15.128185 23558 leader_election.cc:304] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 12589c3399b641fc80f39058a475f8a1, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:15.129106 24519 raft_consensus.cc:2749] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.165089 24519 raft_consensus.cc:493] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:15.165638 24519 raft_consensus.cc:515] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:15.167608 24519 leader_election.cc:290] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
W20260709 13:07:15.173391 23558 leader_election.cc:336] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
W20260709 13:07:15.176694 23556 leader_election.cc:336] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111)
I20260709 13:07:15.177206 23556 leader_election.cc:304] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:15.178023 24519 raft_consensus.cc:2749] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.178761 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:15.179611 13967 tablet_replica.cc:333] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
W20260709 13:07:15.180114 23556 consensus_peers.cc:597] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e -> Peer 12589c3399b641fc80f39058a475f8a1 (127.13.163.195:36741): Couldn't send request to peer 12589c3399b641fc80f39058a475f8a1. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.195:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:15.180655 13967 raft_consensus.cc:2243] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.181641 13967 raft_consensus.cc:2272] T 787a4af7968d4c359c9efbc9abfe9b40 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.183624 13967 tablet_replica.cc:333] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.184235 13967 raft_consensus.cc:2243] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.184623 13967 raft_consensus.cc:2272] T 3038aa8132294ba7832fad08934a848f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.186760 13967 tablet_replica.cc:333] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.187366 13967 raft_consensus.cc:2243] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.187836 13967 raft_consensus.cc:2272] T 6f3067bf3f594c4985b356c81a410f6c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.189874 13967 tablet_replica.cc:333] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.190407 13967 raft_consensus.cc:2243] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.190765 13967 raft_consensus.cc:2272] T 90f7c7f831d14ec5870cb93e9b1f9f02 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.192593 13967 tablet_replica.cc:333] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.192965 24519 raft_consensus.cc:493] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:15.193312 13967 raft_consensus.cc:2243] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.193609 24519 raft_consensus.cc:515] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:15.194490 13967 raft_consensus.cc:2272] T b57c381a872f4c7aae7b11e12cc439b8 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.195822 24519 leader_election.cc:290] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:15.197635 13967 tablet_replica.cc:333] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.198495 13967 raft_consensus.cc:2243] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:07:15.198846 23558 leader_election.cc:336] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:15.199546 13967 raft_consensus.cc:2272] T ce891b8160664205892014cec10858ff P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:15.201562 23556 leader_election.cc:336] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111)
I20260709 13:07:15.201989 23556 leader_election.cc:304] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:15.202442 13967 tablet_replica.cc:333] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.202607 24519 raft_consensus.cc:2749] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.203166 13967 raft_consensus.cc:2243] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.204028 13967 raft_consensus.cc:2272] T 31f1ce58a33542c395d1e4e41fe71ea7 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.206247 13967 tablet_replica.cc:333] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.206878 13967 raft_consensus.cc:2243] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.207347 13967 raft_consensus.cc:2272] T 0c5c2c534bbd4c909a5d8fd8e8ce52e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.209532 13967 tablet_replica.cc:333] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.210086 13967 raft_consensus.cc:2243] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.210556 13967 raft_consensus.cc:2272] T 4743addeb1314388b9469d11e0ff9385 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.212538 13967 tablet_replica.cc:333] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.213235 13967 raft_consensus.cc:2243] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.214042 13967 raft_consensus.cc:2272] T 376a2904d9ff484195a63051a29b8a0d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.215938 13967 tablet_replica.cc:333] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.216475 13967 raft_consensus.cc:2243] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.217361 13967 raft_consensus.cc:2272] T 35d1d7ecd2e941d289aefe4228c0e673 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.219314 13967 tablet_replica.cc:333] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.219848 13967 raft_consensus.cc:2243] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.220615 13967 raft_consensus.cc:2272] T 0ce2a928268b4d73afa32a47bbbcbd64 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.222651 13967 tablet_replica.cc:333] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.223208 13967 raft_consensus.cc:2243] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.223755 13967 raft_consensus.cc:2272] T 0664d2cef22d4b9fbcce815319e6e56c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.226487 13967 tablet_replica.cc:333] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.227327 13967 raft_consensus.cc:2243] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.227862 13967 raft_consensus.cc:2272] T ff658c154728474ab62925bcfa9e4fae P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.229817 13967 tablet_replica.cc:333] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.230406 13967 raft_consensus.cc:2243] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.230870 13967 raft_consensus.cc:2272] T f7d59103647944ec805367e074fecea6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.232682 13967 tablet_replica.cc:333] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.233210 13967 raft_consensus.cc:2243] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.233688 13967 raft_consensus.cc:2272] T f954a7fd7b3840a9b646d16a33cab480 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.235700 13967 tablet_replica.cc:333] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.236232 13967 raft_consensus.cc:2243] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.236656 13967 raft_consensus.cc:2272] T 0675ebf2d38f494d9b749b40e54bea4c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.238600 13967 tablet_replica.cc:333] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.239136 13967 raft_consensus.cc:2243] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.239933 13967 raft_consensus.cc:2272] T 7d1599ca0cb14ed39d2ffde42649f5e6 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.241910 13967 tablet_replica.cc:333] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.242463 13967 raft_consensus.cc:2243] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.243269 13967 raft_consensus.cc:2272] T 0108a896781443dbaf9caef1e04e5cfa P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.245328 13967 tablet_replica.cc:333] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.245903 13967 raft_consensus.cc:2243] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.246675 13967 raft_consensus.cc:2272] T 56c0cd2ce17e48709b4a82c121539b62 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.248586 13967 tablet_replica.cc:333] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.249225 13967 raft_consensus.cc:2243] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.249647 13967 raft_consensus.cc:2272] T fc4a932d15eb4e9e89e5b9bf6627de2b P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.252156 13967 tablet_replica.cc:333] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.252945 13967 raft_consensus.cc:2243] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.253935 13967 raft_consensus.cc:2272] T c366e593434e49a9b0f8d3a41616077d P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.256314 13967 tablet_replica.cc:333] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.256922 13967 raft_consensus.cc:2243] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.257352 13967 raft_consensus.cc:2272] T 21203c7d4b1148e1a4e659daf1df428c P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.259143 13967 tablet_replica.cc:333] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.259697 13967 raft_consensus.cc:2243] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.260118 13967 raft_consensus.cc:2272] T 70d9e995f5e94779b4b8d0a4f921a35c P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.262130 13967 tablet_replica.cc:333] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.262676 13967 raft_consensus.cc:2243] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.263440 13967 raft_consensus.cc:2272] T 893c1c5680b947488f9d810e532fe7e5 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.265476 13967 tablet_replica.cc:333] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.266004 13967 raft_consensus.cc:2243] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.266438 13967 raft_consensus.cc:2272] T 8678c528777f43c7b3552d5d8f484677 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.268373 13967 tablet_replica.cc:333] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.268993 13967 raft_consensus.cc:2243] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.269418 13967 raft_consensus.cc:2272] T d1dd65a066e84285a82713f6ee51aa7e P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.271358 13967 tablet_replica.cc:333] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.271906 13967 raft_consensus.cc:2243] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.272332 13967 raft_consensus.cc:2272] T 5e66e5d5789d4dc3a56f76f34b603bad P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.272863 24519 raft_consensus.cc:493] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d60ed2fc1ba64596942b6df1bf93b4ca)
I20260709 13:07:15.273361 24519 raft_consensus.cc:515] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb56da8fd5145648a898f5487219c6e" member_type: VOTER last_known_addr { host: "127.13.163.196" port: 41791 } } peers { permanent_uuid: "d60ed2fc1ba64596942b6df1bf93b4ca" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35013 } } peers { permanent_uuid: "643f0d55f5ac4308a9f1d5e7158b43fe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 37247 } }
I20260709 13:07:15.274937 13967 tablet_replica.cc:333] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.275557 24519 leader_election.cc:290] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013), 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247)
I20260709 13:07:15.275805 13967 raft_consensus.cc:2243] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.276394 13967 raft_consensus.cc:2272] T cc86fab625684e049569b12b52fdac33 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.279273 13967 tablet_replica.cc:333] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
W20260709 13:07:15.279879 23556 leader_election.cc:336] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 643f0d55f5ac4308a9f1d5e7158b43fe (127.13.163.194:37247): Network error: Client connection negotiation failed: client connection to 127.13.163.194:37247: connect: Connection refused (error 111)
I20260709 13:07:15.280020 13967 raft_consensus.cc:2243] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:15.280423 23558 leader_election.cc:336] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d60ed2fc1ba64596942b6df1bf93b4ca (127.13.163.193:35013): Network error: Client connection negotiation failed: client connection to 127.13.163.193:35013: connect: Connection refused (error 111)
I20260709 13:07:15.280643 13967 raft_consensus.cc:2272] T 1c16a9c7d17249d98740f56a32c03d84 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.280810 23558 leader_election.cc:304] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb56da8fd5145648a898f5487219c6e; no voters: 643f0d55f5ac4308a9f1d5e7158b43fe, d60ed2fc1ba64596942b6df1bf93b4ca
I20260709 13:07:15.281605 24519 raft_consensus.cc:2749] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.283202 13967 tablet_replica.cc:333] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.283776 13967 raft_consensus.cc:2243] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.284197 13967 raft_consensus.cc:2272] T 5fa87a90cab0475da44d21f725d485ab P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.286279 13967 tablet_replica.cc:333] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.286808 13967 raft_consensus.cc:2243] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.287156 13967 raft_consensus.cc:2272] T 37d36e08707f4d8295154d67b1450b0f P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.289029 13967 tablet_replica.cc:333] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.289448 13967 raft_consensus.cc:2243] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.289860 13967 raft_consensus.cc:2272] T 3a3cf23d64f04f4cb98bbd43988743ea P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.291673 13967 tablet_replica.cc:333] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.292109 13967 raft_consensus.cc:2243] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.292511 13967 raft_consensus.cc:2272] T 538d207f493d4b14b97bce7658551672 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.294658 13967 tablet_replica.cc:333] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.295229 13967 raft_consensus.cc:2243] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.295796 13967 raft_consensus.cc:2272] T ec789761c5344074974b9c23b79e418a P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.298477 13967 tablet_replica.cc:333] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.299125 13967 raft_consensus.cc:2243] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.299539 13967 raft_consensus.cc:2272] T 4d8c839e30fa42ee9ff04b9ba759fc85 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.301460 13967 tablet_replica.cc:333] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.301934 13967 raft_consensus.cc:2243] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.302317 13967 raft_consensus.cc:2272] T 8118adb799df4bc2adcd22faba9e44d6 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.304112 13967 tablet_replica.cc:333] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.304531 13967 raft_consensus.cc:2243] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.305341 13967 raft_consensus.cc:2272] T 0ddf62d6f4e045f8b06e7170496fda73 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.307229 13967 tablet_replica.cc:333] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.307703 13967 raft_consensus.cc:2243] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.308072 13967 raft_consensus.cc:2272] T d1fb77caf9eb4699b2319185288758e0 P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.310175 13967 tablet_replica.cc:333] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.310704 13967 raft_consensus.cc:2243] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.311151 13967 raft_consensus.cc:2272] T e1e2b1695b1849db90d73f0c6719c9d3 P ceb56da8fd5145648a898f5487219c6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.313226 13967 tablet_replica.cc:333] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.313798 13967 raft_consensus.cc:2243] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.314559 13967 raft_consensus.cc:2272] T be3f330957084b66847db5f71ee7bc36 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.316318 13967 tablet_replica.cc:333] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.316790 13967 raft_consensus.cc:2243] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.317305 13967 raft_consensus.cc:2272] T 9af1e469bb524d288ab7911185a018b2 P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.319253 13967 tablet_replica.cc:333] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.319744 13967 raft_consensus.cc:2243] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:15.320127 13967 raft_consensus.cc:2272] T 77c7e59145ae42669dbe05a9943aa20e P ceb56da8fd5145648a898f5487219c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.322167 13967 tablet_replica.cc:333] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e: stopping tablet replica
I20260709 13:07:15.322645 13967 raft_consensus.cc:2243] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.323387 13967 raft_consensus.cc:2272] T c4bb4f04978d43798dc3c00c13d510be P ceb56da8fd5145648a898f5487219c6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.395566 13967 tablet_server.cc:196] TabletServer@127.13.163.196:0 shutdown complete.
I20260709 13:07:15.468022 13967 master.cc:562] Master@127.13.163.254:39475 shutting down...
I20260709 13:07:15.495864 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:15.496606 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:15.497092 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3a5a5c29c885437f816972f4b29bdccf: stopping tablet replica
I20260709 13:07:15.520520 13967 master.cc:584] Master@127.13.163.254:39475 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31592 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:07:15.618489 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:38747
I20260709 13:07:15.619724 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:15.626433 24523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.626484 24522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.627578 24525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:15.627688 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:15.628899 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:15.629151 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:15.629319 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602435629302 us; error 0 us; skew 500 ppm
I20260709 13:07:15.629861 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:15.632323 13967 webserver.cc:533] Webserver started at http://127.13.163.254:35177/ using document root <none> and password file <none>
I20260709 13:07:15.632922 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:15.633122 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:15.633415 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:15.634685 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "53d4731a4fb54603a7c2f222c83c6028"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.639576 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:15.642987 24530 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.643867 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:15.644165 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "53d4731a4fb54603a7c2f222c83c6028"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.644460 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:15.681528 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:15.682806 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:15.721863 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:38747
I20260709 13:07:15.721966 24581 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:38747 every 8 connection(s)
I20260709 13:07:15.725947 24582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:15.737116 24582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028: Bootstrap starting.
I20260709 13:07:15.742214 24582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:15.746932 24582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028: No bootstrap required, opened a new log
I20260709 13:07:15.749228 24582 raft_consensus.cc:359] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER }
I20260709 13:07:15.749629 24582 raft_consensus.cc:385] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:15.749881 24582 raft_consensus.cc:740] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d4731a4fb54603a7c2f222c83c6028, State: Initialized, Role: FOLLOWER
I20260709 13:07:15.750584 24582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER }
I20260709 13:07:15.751045 24582 raft_consensus.cc:399] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:15.751339 24582 raft_consensus.cc:493] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:15.751614 24582 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:15.756580 24582 raft_consensus.cc:515] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER }
I20260709 13:07:15.757211 24582 leader_election.cc:304] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53d4731a4fb54603a7c2f222c83c6028; no voters: 
I20260709 13:07:15.758419 24582 leader_election.cc:290] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:15.758932 24585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:15.760361 24585 raft_consensus.cc:697] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 1 LEADER]: Becoming Leader. State: Replica: 53d4731a4fb54603a7c2f222c83c6028, State: Running, Role: LEADER
I20260709 13:07:15.761152 24585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER }
I20260709 13:07:15.761416 24582 sys_catalog.cc:565] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:15.767144 24587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53d4731a4fb54603a7c2f222c83c6028. Latest consensus state: current_term: 1 leader_uuid: "53d4731a4fb54603a7c2f222c83c6028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER } }
I20260709 13:07:15.767930 24587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:15.767805 24586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53d4731a4fb54603a7c2f222c83c6028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d4731a4fb54603a7c2f222c83c6028" member_type: VOTER } }
I20260709 13:07:15.768738 24586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:15.771104 24594 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:15.776450 24594 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:15.781497 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:15.786015 24594 catalog_manager.cc:1383] Generated new cluster ID: e70148a0c4774dac9c76f9c9f98edb77
I20260709 13:07:15.786327 24594 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:15.808593 24594 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:15.809937 24594 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:15.821789 24594 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028: Generated new TSK 0
I20260709 13:07:15.822402 24594 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:15.849500 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:15.856170 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.857116 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.859107 24606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:15.859814 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:15.860775 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:15.861071 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:15.861249 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602435861216 us; error 0 us; skew 500 ppm
I20260709 13:07:15.861820 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:15.864377 13967 webserver.cc:533] Webserver started at http://127.13.163.193:34849/ using document root <none> and password file <none>
I20260709 13:07:15.864976 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:15.865185 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:15.865480 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:15.866595 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "8ffac5c72add4faeadaea45e0591fb40"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.871322 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:15.874972 24611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.875769 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:15.876120 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "8ffac5c72add4faeadaea45e0591fb40"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.876425 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:15.886819 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:15.888104 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:15.889698 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:15.892004 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:15.892230 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.892483 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:15.892673 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.932971 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:33301
I20260709 13:07:15.933060 24673 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:33301 every 8 connection(s)
I20260709 13:07:15.939082 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:15.945956 24678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.946799 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:15.949999 24681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:15.955982 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:15.957171 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:15.957392 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:15.957578 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602435957560 us; error 0 us; skew 500 ppm
I20260709 13:07:15.958009 24674 heartbeater.cc:344] Connected to a master server at 127.13.163.254:38747
I20260709 13:07:15.958304 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:15.958458 24674 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:15.959501 24674 heartbeater.cc:507] Master 127.13.163.254:38747 requested a full tablet report, sending...
I20260709 13:07:15.961293 13967 webserver.cc:533] Webserver started at http://127.13.163.194:33063/ using document root <none> and password file <none>
I20260709 13:07:15.961889 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:15.962091 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:15.961913 24547 ts_manager.cc:194] Registered new tserver with Master: 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301)
I20260709 13:07:15.962404 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:15.963663 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "66f84d8c08484106af287bd611bc5730"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.963953 24547 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59098
I20260709 13:07:15.968969 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.006s
I20260709 13:07:15.972575 24686 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.973446 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:15.973747 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "66f84d8c08484106af287bd611bc5730"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:15.974004 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:15.992750 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:15.994092 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:15.995571 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:15.997789 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:15.997990 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:15.998231 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:15.998386 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:16.039685 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:32851
I20260709 13:07:16.039808 24748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:32851 every 8 connection(s)
I20260709 13:07:16.044651 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:16.053133 24752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:16.053495 24753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:16.056707 13967 server_base.cc:1061] running on GCE node
W20260709 13:07:16.056571 24755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:16.057456 24749 heartbeater.cc:344] Connected to a master server at 127.13.163.254:38747
I20260709 13:07:16.057900 24749 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:16.057956 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:16.058324 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:16.058511 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602436058476 us; error 0 us; skew 500 ppm
I20260709 13:07:16.058790 24749 heartbeater.cc:507] Master 127.13.163.254:38747 requested a full tablet report, sending...
I20260709 13:07:16.059192 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:16.060972 24547 ts_manager.cc:194] Registered new tserver with Master: 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.062470 13967 webserver.cc:533] Webserver started at http://127.13.163.195:36291/ using document root <none> and password file <none>
I20260709 13:07:16.062587 24547 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59108
I20260709 13:07:16.063270 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:16.063570 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:16.064009 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:16.065403 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "ada50d39a793410cb2f6ecde5e0acbae"
format_stamp: "Formatted at 2026-07-09 13:07:16 on dist-test-slave-w3b1"
I20260709 13:07:16.069968 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:07:16.073364 24760 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:16.074146 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:07:16.074460 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "ada50d39a793410cb2f6ecde5e0acbae"
format_stamp: "Formatted at 2026-07-09 13:07:16 on dist-test-slave-w3b1"
I20260709 13:07:16.074817 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:16.101419 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:16.102779 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:16.104326 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:16.106647 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:16.106891 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:16.107136 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:16.107306 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:16.149591 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:37769
I20260709 13:07:16.149717 24822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:37769 every 8 connection(s)
I20260709 13:07:16.164367 24823 heartbeater.cc:344] Connected to a master server at 127.13.163.254:38747
I20260709 13:07:16.164915 24823 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:16.165663 24823 heartbeater.cc:507] Master 127.13.163.254:38747 requested a full tablet report, sending...
I20260709 13:07:16.167810 24547 ts_manager.cc:194] Registered new tserver with Master: ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.168711 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015810748s
I20260709 13:07:16.169391 24547 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59110
I20260709 13:07:16.196249 24547 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59126:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:16.199309 24547 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:16.274736 24788 tablet_service.cc:1511] Processing CreateTablet for tablet 25b6ed18ec1444c899ca33345411f491 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:16.276298 24788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b6ed18ec1444c899ca33345411f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.277913 24787 tablet_service.cc:1511] Processing CreateTablet for tablet 1571d44d0df046f6b2db3f60b400d192 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:16.279388 24787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1571d44d0df046f6b2db3f60b400d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.282913 24786 tablet_service.cc:1511] Processing CreateTablet for tablet fc2bcf9425744c3c96f9ba223fa12da3 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:16.284382 24786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bcf9425744c3c96f9ba223fa12da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.287014 24785 tablet_service.cc:1511] Processing CreateTablet for tablet dd4301a6a9cb41b88e923d3bab6f4871 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:16.288511 24785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4301a6a9cb41b88e923d3bab6f4871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.295228 24784 tablet_service.cc:1511] Processing CreateTablet for tablet beeeceddef6941b88cc25e1a802568f1 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:16.296787 24784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beeeceddef6941b88cc25e1a802568f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.300566 24782 tablet_service.cc:1511] Processing CreateTablet for tablet 521301c9bcc64efb8a6f5069fedc3a05 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:16.303053 24782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521301c9bcc64efb8a6f5069fedc3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.304508 24639 tablet_service.cc:1511] Processing CreateTablet for tablet 25b6ed18ec1444c899ca33345411f491 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:16.305931 24639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b6ed18ec1444c899ca33345411f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.298683 24783 tablet_service.cc:1511] Processing CreateTablet for tablet b15e8dbfb50f4feabc201757f9db629f (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:16.310353 24783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e8dbfb50f4feabc201757f9db629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.310652 24781 tablet_service.cc:1511] Processing CreateTablet for tablet 5db90ee8362143fe9d5a6e9e91591b75 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:16.312041 24781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db90ee8362143fe9d5a6e9e91591b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.314306 24638 tablet_service.cc:1511] Processing CreateTablet for tablet 1571d44d0df046f6b2db3f60b400d192 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:16.315826 24638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1571d44d0df046f6b2db3f60b400d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.318265 24637 tablet_service.cc:1511] Processing CreateTablet for tablet fc2bcf9425744c3c96f9ba223fa12da3 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:16.319819 24634 tablet_service.cc:1511] Processing CreateTablet for tablet b15e8dbfb50f4feabc201757f9db629f (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:16.320699 24636 tablet_service.cc:1511] Processing CreateTablet for tablet dd4301a6a9cb41b88e923d3bab6f4871 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:16.322314 24633 tablet_service.cc:1511] Processing CreateTablet for tablet 521301c9bcc64efb8a6f5069fedc3a05 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:16.323097 24635 tablet_service.cc:1511] Processing CreateTablet for tablet beeeceddef6941b88cc25e1a802568f1 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:16.323248 24714 tablet_service.cc:1511] Processing CreateTablet for tablet 25b6ed18ec1444c899ca33345411f491 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:16.324326 24712 tablet_service.cc:1511] Processing CreateTablet for tablet fc2bcf9425744c3c96f9ba223fa12da3 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:16.326082 24630 tablet_service.cc:1511] Processing CreateTablet for tablet 1638e127d7944b8a9675e92811ac8db6 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:16.326246 24713 tablet_service.cc:1511] Processing CreateTablet for tablet 1571d44d0df046f6b2db3f60b400d192 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:16.324716 24714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b6ed18ec1444c899ca33345411f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.329010 24637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bcf9425744c3c96f9ba223fa12da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.328195 24711 tablet_service.cc:1511] Processing CreateTablet for tablet dd4301a6a9cb41b88e923d3bab6f4871 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:16.329648 24631 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce8ce5e5cc44212a8163ba44d266611 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:16.330149 24712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2bcf9425744c3c96f9ba223fa12da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.330302 24710 tablet_service.cc:1511] Processing CreateTablet for tablet beeeceddef6941b88cc25e1a802568f1 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:16.331108 24631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce8ce5e5cc44212a8163ba44d266611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.331806 24710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beeeceddef6941b88cc25e1a802568f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.333199 24633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521301c9bcc64efb8a6f5069fedc3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.335527 24636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4301a6a9cb41b88e923d3bab6f4871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.324738 24632 tablet_service.cc:1511] Processing CreateTablet for tablet 5db90ee8362143fe9d5a6e9e91591b75 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:16.337203 24635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beeeceddef6941b88cc25e1a802568f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.338007 24632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db90ee8362143fe9d5a6e9e91591b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.337636 24709 tablet_service.cc:1511] Processing CreateTablet for tablet b15e8dbfb50f4feabc201757f9db629f (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:16.338220 24711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4301a6a9cb41b88e923d3bab6f4871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.339073 24709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e8dbfb50f4feabc201757f9db629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.339673 24630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1638e127d7944b8a9675e92811ac8db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.340435 24634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e8dbfb50f4feabc201757f9db629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.341049 24713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1571d44d0df046f6b2db3f60b400d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.364281 24708 tablet_service.cc:1511] Processing CreateTablet for tablet 521301c9bcc64efb8a6f5069fedc3a05 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:16.365854 24708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521301c9bcc64efb8a6f5069fedc3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.366376 24780 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce8ce5e5cc44212a8163ba44d266611 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:16.367774 24780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce8ce5e5cc44212a8163ba44d266611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.370087 24707 tablet_service.cc:1511] Processing CreateTablet for tablet 5db90ee8362143fe9d5a6e9e91591b75 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:16.370447 24779 tablet_service.cc:1511] Processing CreateTablet for tablet 1638e127d7944b8a9675e92811ac8db6 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:16.371587 24707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db90ee8362143fe9d5a6e9e91591b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.371292 24705 tablet_service.cc:1511] Processing CreateTablet for tablet 1638e127d7944b8a9675e92811ac8db6 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:16.371775 24779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1638e127d7944b8a9675e92811ac8db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.372622 24705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1638e127d7944b8a9675e92811ac8db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.373790 24706 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce8ce5e5cc44212a8163ba44d266611 (DEFAULT_TABLE table=test-workload [id=ebd1b42e81c747b4bb1eb703fe85d312]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:16.375188 24706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce8ce5e5cc44212a8163ba44d266611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:16.396716 24838 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.417147 24839 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.426829 24838 tablet_bootstrap.cc:654] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.432789 24838 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.433401 24838 ts_tablet_manager.cc:1403] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.007s
I20260709 13:07:16.436407 24838 raft_consensus.cc:359] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.437270 24838 raft_consensus.cc:385] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.437608 24838 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.438427 24838 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.449141 24674 heartbeater.cc:499] Master 127.13.163.254:38747 was elected leader, sending a full tablet report...
I20260709 13:07:16.449561 24838 ts_tablet_manager.cc:1434] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:07:16.450748 24838 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.459175 24838 tablet_bootstrap.cc:654] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.499184 24839 tablet_bootstrap.cc:654] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.514531 24840 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:16.569267 24842 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.569916 24842 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.570420 24840 tablet_bootstrap.cc:654] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.586066 24842 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.595247 24839 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.596058 24839 ts_tablet_manager.cc:1403] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.179s	user 0.029s	sys 0.038s
I20260709 13:07:16.600221 24839 raft_consensus.cc:359] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.601284 24839 raft_consensus.cc:385] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.601665 24839 raft_consensus.cc:740] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.602911 24839 consensus_queue.cc:260] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.603324 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
W20260709 13:07:16.606195 24614 leader_election.cc:343] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.610347 24838 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.611011 24838 ts_tablet_manager.cc:1403] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.160s	user 0.058s	sys 0.091s
I20260709 13:07:16.610839 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:16.612255 24840 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
W20260709 13:07:16.612669 24615 leader_election.cc:343] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.613065 24840 ts_tablet_manager.cc:1403] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.012s
I20260709 13:07:16.613209 24615 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40; no voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:16.614251 24842 raft_consensus.cc:2749] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:16.614195 24838 raft_consensus.cc:359] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.615171 24838 raft_consensus.cc:385] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.615476 24838 raft_consensus.cc:740] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.616302 24838 consensus_queue.cc:260] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.617105 24749 heartbeater.cc:499] Master 127.13.163.254:38747 was elected leader, sending a full tablet report...
I20260709 13:07:16.617450 24840 raft_consensus.cc:359] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.618257 24838 ts_tablet_manager.cc:1434] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:16.618330 24839 ts_tablet_manager.cc:1434] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.008s
I20260709 13:07:16.618289 24840 raft_consensus.cc:385] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.619024 24840 raft_consensus.cc:740] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.619817 24839 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.621064 24838 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.620020 24840 consensus_queue.cc:260] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.627414 24839 tablet_bootstrap.cc:654] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.628144 24838 tablet_bootstrap.cc:654] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.636279 24823 heartbeater.cc:499] Master 127.13.163.254:38747 was elected leader, sending a full tablet report...
I20260709 13:07:16.636454 24840 ts_tablet_manager.cc:1434] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.009s
I20260709 13:07:16.637854 24840 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:16.645344 24840 tablet_bootstrap.cc:654] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.648049 24839 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.648725 24839 ts_tablet_manager.cc:1403] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.004s
I20260709 13:07:16.653443 24839 raft_consensus.cc:359] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:16.653872 24824 tablet.cc:2406] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:16.657368 24839 raft_consensus.cc:385] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.657929 24839 raft_consensus.cc:740] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.659188 24838 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.659920 24838 ts_tablet_manager.cc:1403] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.014s
I20260709 13:07:16.659235 24839 consensus_queue.cc:260] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.663409 24838 raft_consensus.cc:359] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.664217 24838 raft_consensus.cc:385] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.664553 24838 raft_consensus.cc:740] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.664608 24839 ts_tablet_manager.cc:1434] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:07:16.665902 24839 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.665485 24838 consensus_queue.cc:260] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.667876 24838 ts_tablet_manager.cc:1434] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:16.668962 24838 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.672318 24840 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:16.673007 24840 ts_tablet_manager.cc:1403] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.012s
I20260709 13:07:16.673094 24839 tablet_bootstrap.cc:654] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.675868 24838 tablet_bootstrap.cc:654] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.677129 24840 raft_consensus.cc:359] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.678472 24840 raft_consensus.cc:385] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.678910 24840 raft_consensus.cc:740] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.679991 24840 consensus_queue.cc:260] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.683132 24840 ts_tablet_manager.cc:1434] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:07:16.684337 24840 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:16.686065 24839 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.686684 24839 ts_tablet_manager.cc:1403] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 13:07:16.693784 24839 raft_consensus.cc:359] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.694348 24840 tablet_bootstrap.cc:654] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.694702 24839 raft_consensus.cc:385] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.695076 24839 raft_consensus.cc:740] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.696097 24839 consensus_queue.cc:260] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.698988 24839 ts_tablet_manager.cc:1434] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:07:16.700205 24839 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.704562 24838 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.705152 24838 ts_tablet_manager.cc:1403] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.020s
I20260709 13:07:16.707465 24839 tablet_bootstrap.cc:654] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.707369 24838 raft_consensus.cc:359] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.708033 24838 raft_consensus.cc:385] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.708350 24838 raft_consensus.cc:740] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.709561 24838 consensus_queue.cc:260] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.715490 24838 ts_tablet_manager.cc:1434] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:16.716714 24838 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.717355 24840 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:16.718191 24840 ts_tablet_manager.cc:1403] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.011s
I20260709 13:07:16.723675 24839 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.723675 24840 raft_consensus.cc:359] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.724254 24838 tablet_bootstrap.cc:654] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.724653 24842 raft_consensus.cc:493] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.724864 24840 raft_consensus.cc:385] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.724479 24839 ts_tablet_manager.cc:1403] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.016s
I20260709 13:07:16.725694 24840 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.725706 24842 raft_consensus.cc:515] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.726879 24840 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.729230 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.729982 24840 ts_tablet_manager.cc:1434] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 13:07:16.731210 24840 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
W20260709 13:07:16.731287 24614 leader_election.cc:343] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.733214 24839 raft_consensus.cc:359] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.734146 24839 raft_consensus.cc:385] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.734514 24839 raft_consensus.cc:740] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.734683 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:16.735661 24839 consensus_queue.cc:260] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:16.737135 24615 leader_election.cc:343] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.738271 24615 leader_election.cc:304] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40; no voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:16.739357 24840 tablet_bootstrap.cc:654] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.741696 24839 ts_tablet_manager.cc:1434] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 13:07:16.741711 24842 leader_election.cc:290] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.744185 24842 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.744282 24859 raft_consensus.cc:2749] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:16.745085 24842 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.748366 24842 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.752874 24840 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:16.749601 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.753490 24840 ts_tablet_manager.cc:1403] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:07:16.753880 24724 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:16.755380 24614 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:16.743003 24839 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.756685 24840 raft_consensus.cc:359] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.757613 24840 raft_consensus.cc:385] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.757982 24840 raft_consensus.cc:740] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.758942 24840 consensus_queue.cc:260] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.759863 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:16.761737 24840 ts_tablet_manager.cc:1434] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
W20260709 13:07:16.762316 24615 leader_election.cc:343] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.762976 24840 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:16.756937 24858 raft_consensus.cc:2804] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:16.763697 24858 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.764192 24858 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.770438 24840 tablet_bootstrap.cc:654] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.775158 24838 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.775914 24838 ts_tablet_manager.cc:1403] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.059s	user 0.031s	sys 0.014s
W20260709 13:07:16.776458 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:16.779527 24838 raft_consensus.cc:359] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.779554 24858 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.780606 24839 tablet_bootstrap.cc:654] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.780923 24838 raft_consensus.cc:385] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.781459 24838 raft_consensus.cc:740] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.782651 24840 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:16.782459 24838 consensus_queue.cc:260] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.783365 24840 ts_tablet_manager.cc:1403] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.011s
I20260709 13:07:16.783846 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:16.784570 24724 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.785243 24838 ts_tablet_manager.cc:1434] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:07:16.786468 24838 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.788324 24849 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.789031 24849 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.794371 24838 tablet_bootstrap.cc:654] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.796020 24849 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.810473 24724 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 1.
I20260709 13:07:16.787581 24840 raft_consensus.cc:359] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.814095 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:16.814966 24858 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Requested vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.816766 24840 raft_consensus.cc:385] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.817354 24840 raft_consensus.cc:740] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.817677 24614 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
W20260709 13:07:16.819317 24615 leader_election.cc:343] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.820779 24858 raft_consensus.cc:2804] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:16.822808 24858 raft_consensus.cc:697] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Becoming Leader. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Running, Role: LEADER
I20260709 13:07:16.824198 24858 consensus_queue.cc:237] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.821285 24840 consensus_queue.cc:260] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.842353 24849 raft_consensus.cc:493] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.843089 24849 raft_consensus.cc:515] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.846513 24544 catalog_manager.cc:5697] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "8ffac5c72add4faeadaea45e0591fb40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:16.880637 24838 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.881501 24838 ts_tablet_manager.cc:1403] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.095s	user 0.044s	sys 0.049s
I20260709 13:07:16.883658 24849 leader_election.cc:290] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.884665 24840 ts_tablet_manager.cc:1434] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.004s
I20260709 13:07:16.891649 24648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:16.885303 24838 raft_consensus.cc:359] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.887926 24839 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.890690 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:16.893153 24838 raft_consensus.cc:385] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.893481 24838 raft_consensus.cc:740] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.893441 24649 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
I20260709 13:07:16.894392 24838 consensus_queue.cc:260] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.894620 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.895435 24839 ts_tablet_manager.cc:1403] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.153s	user 0.048s	sys 0.066s
I20260709 13:07:16.897250 24838 ts_tablet_manager.cc:1434] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:07:16.898407 24838 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.898593 24839 raft_consensus.cc:359] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.899369 24839 raft_consensus.cc:385] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:16.894737 24763 leader_election.cc:343] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.894526 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.899700 24839 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.900069 24723 raft_consensus.cc:2468] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
W20260709 13:07:16.900418 24763 leader_election.cc:343] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.900552 24839 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.901163 24763 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730
I20260709 13:07:16.902530 24763 leader_election.cc:304] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:16.903065 24839 ts_tablet_manager.cc:1434] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:16.904174 24839 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.904750 24840 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:16.905493 24838 tablet_bootstrap.cc:654] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.909268 24849 raft_consensus.cc:2804] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:16.909807 24849 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.910437 24849 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.911413 24839 tablet_bootstrap.cc:654] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.922550 24840 tablet_bootstrap.cc:654] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:16.922916 24869 raft_consensus.cc:2804] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:16.929093 24869 raft_consensus.cc:493] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.929563 24869 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.933370 24838 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:16.933940 24838 ts_tablet_manager.cc:1403] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.017s
I20260709 13:07:16.936429 24839 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:16.926434 24849 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:16.937150 24839 ts_tablet_manager.cc:1403] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.019s
I20260709 13:07:16.939191 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:16.939857 24649 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.939987 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:16.940652 24723 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.939375 24849 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.943627 24840 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:16.944203 24840 ts_tablet_manager.cc:1403] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:07:16.947453 24840 raft_consensus.cc:359] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.948158 24840 raft_consensus.cc:385] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.948479 24840 raft_consensus.cc:740] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.948598 24723 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:16.949157 24649 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:16.949287 24838 raft_consensus.cc:359] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.950166 24838 raft_consensus.cc:385] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.950013 24763 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:16.950589 24838 raft_consensus.cc:740] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.950722 24840 consensus_queue.cc:260] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.951704 24839 raft_consensus.cc:359] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.951478 24838 consensus_queue.cc:260] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.952701 24839 raft_consensus.cc:385] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:16.953269 24849 raft_consensus.cc:2804] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:16.953270 24839 raft_consensus.cc:740] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:16.955044 24839 consensus_queue.cc:260] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.959780 24839 ts_tablet_manager.cc:1434] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.005s
I20260709 13:07:16.956918 24874 raft_consensus.cc:493] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.963153 24874 raft_consensus.cc:515] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.964273 24838 ts_tablet_manager.cc:1434] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.003s
I20260709 13:07:16.967245 24874 leader_election.cc:290] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.969730 24874 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.971117 24838 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:16.971016 24874 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.967588 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:16.970026 24858 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.972895 24858 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:16.974200 24763 leader_election.cc:343] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.975086 24858 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.968529 24839 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:16.971513 24724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.976716 24848 raft_consensus.cc:493] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.977807 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:16.978031 24848 raft_consensus.cc:515] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:16.978915 24722 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:16.982342 24614 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:16.992264 24874 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:16.981240 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
W20260709 13:07:16.993419 24763 leader_election.cc:343] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:16.996610 24858 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:16.996591 24763 leader_election.cc:304] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:16.996321 24798 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:16.997733 24874 raft_consensus.cc:2749] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:16.997277 24858 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.999387 24858 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:16.999361 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:16.983400 24848 leader_election.cc:290] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:16.976774 24849 raft_consensus.cc:697] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:17.001221 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.001050 24849 consensus_queue.cc:237] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:17.001897 24723 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
I20260709 13:07:17.003165 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:17.004037 24874 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.004419 24874 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.004738 24874 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.007541 24838 tablet_bootstrap.cc:654] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.007977 24858 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.009274 24649 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ada50d39a793410cb2f6ecde5e0acbae in current term 1: Already voted for candidate 8ffac5c72add4faeadaea45e0591fb40 in this term.
I20260709 13:07:17.014273 24869 raft_consensus.cc:515] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.016747 24869 leader_election.cc:290] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.017405 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.020679 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:17.021237 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
W20260709 13:07:17.021554 24763 leader_election.cc:343] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.021864 24723 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.022339 24858 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Requested vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.016786 24839 tablet_bootstrap.cc:654] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.020531 24874 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.026682 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.027367 24722 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.027522 24798 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ffac5c72add4faeadaea45e0591fb40 in current term 1: Already voted for candidate ada50d39a793410cb2f6ecde5e0acbae in this term.
I20260709 13:07:17.027758 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:17.028373 24840 ts_tablet_manager.cc:1434] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.084s	user 0.026s	sys 0.029s
I20260709 13:07:17.028447 24649 raft_consensus.cc:2468] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 0.
I20260709 13:07:17.029881 24840 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:17.029948 24689 leader_election.cc:304] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:17.030913 24848 raft_consensus.cc:2804] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.031316 24848 raft_consensus.cc:493] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.031646 24848 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.032207 24874 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.035440 24722 raft_consensus.cc:2468] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:17.036278 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.037160 24649 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ada50d39a793410cb2f6ecde5e0acbae in current term 1: Already voted for candidate 8ffac5c72add4faeadaea45e0591fb40 in this term.
I20260709 13:07:17.039340 24763 leader_election.cc:304] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:17.039563 24848 raft_consensus.cc:515] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.040714 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:17.041923 24874 raft_consensus.cc:2804] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.042181 24848 leader_election.cc:290] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.042850 24874 raft_consensus.cc:697] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:17.043463 24723 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 1.
I20260709 13:07:17.043747 24874 consensus_queue.cc:237] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.044126 24545 catalog_manager.cc:5697] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 0 to 1, leader changed from <none> to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 1 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.044498 24614 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:17.046451 24858 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.046948 24858 raft_consensus.cc:697] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Becoming Leader. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Running, Role: LEADER
W20260709 13:07:17.046983 24690 leader_election.cc:343] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.048115 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:17.047866 24858 consensus_queue.cc:237] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.047307 24875 raft_consensus.cc:493] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.049206 24875 raft_consensus.cc:515] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.049799 24840 tablet_bootstrap.cc:654] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.050014 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.050810 24722 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ada50d39a793410cb2f6ecde5e0acbae in current term 1: Already voted for candidate 8ffac5c72add4faeadaea45e0591fb40 in this term.
I20260709 13:07:17.050835 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.051388 24875 leader_election.cc:290] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.051730 24649 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:17.056120 24690 leader_election.cc:343] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.058280 24648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:17.058642 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
W20260709 13:07:17.060314 24763 leader_election.cc:343] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.061146 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.061841 24722 raft_consensus.cc:2468] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
I20260709 13:07:17.062951 24763 leader_election.cc:304] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:17.063392 24875 raft_consensus.cc:2749] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:17.063778 24849 raft_consensus.cc:2804] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.064191 24849 raft_consensus.cc:493] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.064524 24849 raft_consensus.cc:3060] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.074046 24840 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:17.074671 24840 ts_tablet_manager.cc:1403] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.013s
I20260709 13:07:17.074790 24838 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:17.075387 24545 catalog_manager.cc:5697] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "8ffac5c72add4faeadaea45e0591fb40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.077361 24649 raft_consensus.cc:2468] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:17.075311 24838 ts_tablet_manager.cc:1403] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.001s
I20260709 13:07:17.078200 24840 raft_consensus.cc:359] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.078773 24689 leader_election.cc:304] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:17.077562 24849 raft_consensus.cc:515] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.080031 24848 raft_consensus.cc:2804] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.081002 24849 leader_election.cc:290] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.080891 24838 raft_consensus.cc:359] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.081884 24838 raft_consensus.cc:385] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.078997 24840 raft_consensus.cc:385] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.082985 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.083397 24840 raft_consensus.cc:740] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.079227 24839 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:17.082237 24838 raft_consensus.cc:740] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.084438 24839 ts_tablet_manager.cc:1403] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.116s	user 0.012s	sys 0.004s
W20260709 13:07:17.084440 24763 leader_election.cc:343] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.085671 24838 consensus_queue.cc:260] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.087956 24838 ts_tablet_manager.cc:1434] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:07:17.087090 24544 catalog_manager.cc:5697] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 0 to 1, leader changed from <none> to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 1 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.089051 24838 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:17.093201 24839 raft_consensus.cc:359] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.095134 24839 raft_consensus.cc:385] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.095328 24838 tablet_bootstrap.cc:654] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.095618 24839 raft_consensus.cc:740] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.094192 24840 consensus_queue.cc:260] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.096640 24839 consensus_queue.cc:260] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.098570 24840 ts_tablet_manager.cc:1434] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:07:17.099581 24840 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:17.100602 24839 ts_tablet_manager.cc:1434] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:07:17.101840 24839 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:17.106015 24838 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:17.106576 24838 ts_tablet_manager.cc:1403] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.002s
I20260709 13:07:17.108462 24839 tablet_bootstrap.cc:654] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.109463 24838 raft_consensus.cc:359] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.110224 24838 raft_consensus.cc:385] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.110532 24838 raft_consensus.cc:740] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.110769 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.111320 24838 consensus_queue.cc:260] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.112749 24840 tablet_bootstrap.cc:654] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.114202 24838 ts_tablet_manager.cc:1434] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:07:17.111364 24722 raft_consensus.cc:3060] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.122298 24838 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:17.125988 24848 raft_consensus.cc:697] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:17.127208 24848 consensus_queue.cc:237] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.133363 24840 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:17.135265 24840 ts_tablet_manager.cc:1403] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.000s
I20260709 13:07:17.137919 24840 raft_consensus.cc:359] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.138592 24840 raft_consensus.cc:385] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.138864 24840 raft_consensus.cc:740] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.139515 24840 consensus_queue.cc:260] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.141494 24840 ts_tablet_manager.cc:1434] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:17.143493 24722 raft_consensus.cc:2468] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:17.146402 24858 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.145053 24840 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:17.147065 24858 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.146349 24544 catalog_manager.cc:5697] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.145928 24838 tablet_bootstrap.cc:654] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.149897 24839 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:17.150441 24839 ts_tablet_manager.cc:1403] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.049s	user 0.037s	sys 0.007s
I20260709 13:07:17.151791 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:17.153162 24839 raft_consensus.cc:359] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.153442 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.153920 24839 raft_consensus.cc:385] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.154248 24839 raft_consensus.cc:740] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.154331 24723 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:17.155017 24839 consensus_queue.cc:260] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:17.156088 24615 leader_election.cc:343] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.157073 24839 ts_tablet_manager.cc:1434] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:17.157552 24614 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:17.158078 24839 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:17.158432 24842 raft_consensus.cc:2804] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.158844 24842 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.159158 24842 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.161511 24858 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.161962 24763 leader_election.cc:304] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:17.163244 24849 raft_consensus.cc:2804] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.163730 24849 raft_consensus.cc:697] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:17.164530 24849 consensus_queue.cc:237] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.165537 24840 tablet_bootstrap.cc:654] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.169083 24842 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.172814 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.173664 24722 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.176700 24839 tablet_bootstrap.cc:654] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.177991 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
W20260709 13:07:17.180141 24615 leader_election.cc:343] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:17.180970 24842 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Requested vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.190723 24545 catalog_manager.cc:5697] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 0 to 1, leader changed from <none> to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 1 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.195626 24839 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:17.195958 24838 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: No bootstrap required, opened a new log
I20260709 13:07:17.195729 24722 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 1.
I20260709 13:07:17.196332 24839 ts_tablet_manager.cc:1403] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.002s
I20260709 13:07:17.196552 24838 ts_tablet_manager.cc:1403] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.075s	user 0.018s	sys 0.019s
I20260709 13:07:17.197607 24614 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:17.198552 24842 raft_consensus.cc:2804] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.199055 24842 raft_consensus.cc:697] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Becoming Leader. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Running, Role: LEADER
I20260709 13:07:17.199246 24839 raft_consensus.cc:359] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.199968 24842 consensus_queue.cc:237] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.200301 24838 raft_consensus.cc:359] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.201125 24838 raft_consensus.cc:385] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.201485 24838 raft_consensus.cc:740] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.202581 24838 consensus_queue.cc:260] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.199987 24839 raft_consensus.cc:385] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.205004 24839 raft_consensus.cc:740] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.205157 24838 ts_tablet_manager.cc:1434] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:17.206820 24839 consensus_queue.cc:260] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.211081 24840 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:17.211460 24839 ts_tablet_manager.cc:1434] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:07:17.211665 24840 ts_tablet_manager.cc:1403] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.017s
I20260709 13:07:17.212513 24839 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: Bootstrap starting.
I20260709 13:07:17.213115 24545 catalog_manager.cc:5697] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "8ffac5c72add4faeadaea45e0591fb40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.214521 24840 raft_consensus.cc:359] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.215274 24840 raft_consensus.cc:385] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.215605 24840 raft_consensus.cc:740] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.218820 24839 tablet_bootstrap.cc:654] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.216527 24840 consensus_queue.cc:260] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.225795 24840 ts_tablet_manager.cc:1434] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:17.226976 24840 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: Bootstrap starting.
I20260709 13:07:17.230121 24839 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: No bootstrap required, opened a new log
I20260709 13:07:17.230731 24839 ts_tablet_manager.cc:1403] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:07:17.233783 24840 tablet_bootstrap.cc:654] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:17.233646 24839 raft_consensus.cc:359] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.234402 24839 raft_consensus.cc:385] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.234730 24839 raft_consensus.cc:740] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.235497 24839 consensus_queue.cc:260] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.237975 24839 ts_tablet_manager.cc:1434] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:17.240748 24840 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: No bootstrap required, opened a new log
I20260709 13:07:17.241274 24840 ts_tablet_manager.cc:1403] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:07:17.243144 24840 raft_consensus.cc:359] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.243650 24840 raft_consensus.cc:385] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:17.243872 24840 raft_consensus.cc:740] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Initialized, Role: FOLLOWER
I20260709 13:07:17.244437 24840 consensus_queue.cc:260] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.246536 24840 ts_tablet_manager.cc:1434] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:17.249158 24842 raft_consensus.cc:493] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.249636 24842 raft_consensus.cc:515] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.251544 24842 leader_election.cc:290] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.252343 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.252756 24798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:17.253185 24722 raft_consensus.cc:2393] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ffac5c72add4faeadaea45e0591fb40 in current term 1: Already voted for candidate ada50d39a793410cb2f6ecde5e0acbae in this term.
I20260709 13:07:17.254477 24615 leader_election.cc:304] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40; no voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:17.255421 24848 raft_consensus.cc:493] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.255852 24848 raft_consensus.cc:515] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.256322 24798 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.257929 24848 leader_election.cc:290] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.258653 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:17.258906 24849 raft_consensus.cc:493] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.259325 24649 raft_consensus.cc:2468] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 0.
I20260709 13:07:17.259394 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:17.259372 24849 raft_consensus.cc:515] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.260339 24797 raft_consensus.cc:2468] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 0.
I20260709 13:07:17.260818 24689 leader_election.cc:304] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:17.262202 24849 leader_election.cc:290] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.262363 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40" is_pre_election: true
I20260709 13:07:17.263012 24649 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
I20260709 13:07:17.263231 24848 raft_consensus.cc:2804] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.263648 24848 raft_consensus.cc:493] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.263705 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.264101 24848 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.264287 24763 leader_election.cc:304] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:17.259550 24842 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.265278 24849 raft_consensus.cc:2804] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.265698 24849 raft_consensus.cc:493] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.266088 24849 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.264438 24722 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 0.
I20260709 13:07:17.270082 24842 raft_consensus.cc:2749] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:17.271799 24848 raft_consensus.cc:515] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.272042 24842 consensus_queue.cc:1048] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:17.274569 24848 leader_election.cc:290] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.274681 24649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.275281 24649 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.275806 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:17.276398 24797 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.283229 24649 raft_consensus.cc:2468] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:17.285254 24689 leader_election.cc:304] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:17.273779 24849 raft_consensus.cc:515] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.286154 24848 raft_consensus.cc:2804] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.286870 24848 raft_consensus.cc:697] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:17.287987 24848 consensus_queue.cc:237] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.289345 24648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:17.289999 24648 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.290792 24797 raft_consensus.cc:2468] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:17.297166 24842 raft_consensus.cc:493] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.300724 24648 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:17.303752 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.304363 24722 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.304601 24763 leader_election.cc:304] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:17.301142 24842 raft_consensus.cc:515] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.305459 24874 raft_consensus.cc:2804] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.305976 24874 raft_consensus.cc:697] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:17.306818 24874 consensus_queue.cc:237] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.302129 24849 leader_election.cc:290] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.307209 24842 leader_election.cc:290] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.311798 24722 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:17.312343 24842 consensus_queue.cc:1048] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:17.318123 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:17.319095 24797 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:17.320659 24615 leader_election.cc:304] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:17.322117 24903 raft_consensus.cc:2804] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:17.322676 24903 raft_consensus.cc:493] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.323146 24903 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.327209 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.328048 24722 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 0.
I20260709 13:07:17.330806 24545 catalog_manager.cc:5697] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 0 to 1, leader changed from <none> to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 1 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.331966 24903 raft_consensus.cc:515] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.331933 24544 catalog_manager.cc:5697] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.335631 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:17.336280 24796 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.340224 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:17.341120 24722 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.344388 24796 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 1.
I20260709 13:07:17.346223 24615 leader_election.cc:304] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:17.348402 24842 raft_consensus.cc:2804] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:17.348953 24842 raft_consensus.cc:697] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Becoming Leader. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Running, Role: LEADER
I20260709 13:07:17.349815 24842 consensus_queue.cc:237] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:17.344951 24903 leader_election.cc:290] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 election: Requested vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:17.352629 24722 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ffac5c72add4faeadaea45e0591fb40 in term 1.
I20260709 13:07:17.369498 24544 catalog_manager.cc:5697] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "8ffac5c72add4faeadaea45e0591fb40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.409915 13967 ts_manager.cc:295] Set tserver state for 8ffac5c72add4faeadaea45e0591fb40 to MAINTENANCE_MODE
I20260709 13:07:17.410858 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
I20260709 13:07:17.432260 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:17.433132 13967 tablet_replica.cc:333] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.434017 13967 raft_consensus.cc:2243] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.434388 13967 raft_consensus.cc:2272] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.436491 13967 tablet_replica.cc:333] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.437162 13967 raft_consensus.cc:2243] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.437534 13967 raft_consensus.cc:2272] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.439685 13967 tablet_replica.cc:333] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.440423 13967 raft_consensus.cc:2243] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:17.441183 13967 pending_rounds.cc:70] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Trying to abort 1 pending ops.
I20260709 13:07:17.441471 13967 pending_rounds.cc:77] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635583400693760 op_type: NO_OP noop_request { }
I20260709 13:07:17.441979 13967 raft_consensus.cc:2889] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:17.442343 13967 raft_consensus.cc:2272] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.442780 24842 consensus_queue.cc:1048] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:17.443946 24763 consensus_peers.cc:597] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.445232 13967 tablet_replica.cc:333] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.445987 13967 raft_consensus.cc:2243] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.446445 13967 raft_consensus.cc:2272] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.449506 13967 tablet_replica.cc:333] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.450281 13967 raft_consensus.cc:2243] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.450794 13967 raft_consensus.cc:2272] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.453940 13967 tablet_replica.cc:333] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.454298 24842 consensus_queue.cc:1048] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:17.454784 13967 raft_consensus.cc:2243] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:17.455981 13967 raft_consensus.cc:2272] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.459023 13967 tablet_replica.cc:333] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.460008 13967 raft_consensus.cc:2243] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.460561 13967 raft_consensus.cc:2272] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.463033 13967 tablet_replica.cc:333] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.463768 13967 raft_consensus.cc:2243] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:17.464779 13967 raft_consensus.cc:2272] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.466967 13967 tablet_replica.cc:333] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.467511 13967 raft_consensus.cc:2243] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:17.468196 13967 pending_rounds.cc:70] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Trying to abort 1 pending ops.
I20260709 13:07:17.468380 13967 pending_rounds.cc:77] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635582785581056 op_type: NO_OP noop_request { }
I20260709 13:07:17.468703 13967 raft_consensus.cc:2889] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:17.469040 13967 raft_consensus.cc:2272] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:17.471402 13967 tablet_replica.cc:333] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: stopping tablet replica
I20260709 13:07:17.471910 13967 raft_consensus.cc:2243] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:17.472292 13967 raft_consensus.cc:2272] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:17.487097 13967 connection.cc:466] server connection from 127.0.0.1:48604 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48604 (request call id 13) could send its response
I20260709 13:07:17.488700 24874 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.489295 24874 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:17.490078 24763 consensus_peers.cc:597] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.491703 24874 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.493181 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:17.494249 24723 raft_consensus.cc:2393] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ada50d39a793410cb2f6ecde5e0acbae in current term 1: Already voted for candidate 8ffac5c72add4faeadaea45e0591fb40 in this term.
I20260709 13:07:17.495888 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
W20260709 13:07:17.496058 24763 leader_election.cc:336] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:17.496564 24763 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:17.497293 24874 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.502460 24874 raft_consensus.cc:2749] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:17.530297 24874 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:17.540391 24689 consensus_peers.cc:597] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.564293 24849 consensus_queue.cc:1048] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:17.608675 24763 consensus_peers.cc:597] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.644413 24849 consensus_queue.cc:1048] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:17.675377 24848 consensus_queue.cc:1048] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:17.686239 24849 raft_consensus.cc:493] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.686759 24849 raft_consensus.cc:515] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:17.688323 24689 consensus_peers.cc:597] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.688898 24849 leader_election.cc:290] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:17.691390 24723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
W20260709 13:07:17.693866 24763 leader_election.cc:336] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:17.694355 24763 leader_election.cc:304] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:17.695098 24849 raft_consensus.cc:2749] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:17.704252 24763 consensus_peers.cc:597] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.725039 24797 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:17.731791 24918 consensus_queue.cc:1048] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:17.777550 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:17.790911 24849 consensus_queue.cc:1048] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:18.778626 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:18.784940 24921 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.785838 24921 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.788736 24921 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
W20260709 13:07:18.790482 24763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111) [suppressed 147 similar messages]
I20260709 13:07:18.791229 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
W20260709 13:07:18.793759 24763 leader_election.cc:336] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.794313 24763 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:18.795012 24921 raft_consensus.cc:2749] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:18.826407 24886 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.827283 24886 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.830288 24886 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.831676 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:18.832571 24797 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:18.835078 24690 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:18.835831 24886 raft_consensus.cc:2804] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:18.835901 24689 leader_election.cc:336] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.836256 24886 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.836772 24886 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.843884 24886 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.845947 24886 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.847174 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:18.847851 24797 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:18.849840 24689 leader_election.cc:336] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.853446 24797 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:18.854537 24690 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:18.855203 24886 raft_consensus.cc:2804] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:18.856096 24886 raft_consensus.cc:697] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:18.856976 24886 consensus_queue.cc:237] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.864213 24544 catalog_manager.cc:5697] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:18.936583 24886 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.937217 24886 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.939079 24886 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.940734 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:18.941483 24797 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:18.942724 24690 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
W20260709 13:07:18.943218 24689 leader_election.cc:336] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.943413 24886 raft_consensus.cc:2804] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:18.943861 24886 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.944167 24886 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.949222 24886 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.950801 24886 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.951848 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:18.952382 24797 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:18.954739 24689 leader_election.cc:336] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.954900 24886 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.955498 24886 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.957516 24886 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.958410 24921 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.958773 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:18.959030 24921 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.959290 24797 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:18.960517 24690 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:18.960642 24796 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:18.961413 24921 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
W20260709 13:07:18.962648 24689 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.961720 24886 raft_consensus.cc:2804] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:18.963446 24886 raft_consensus.cc:697] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:18.963671 24690 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:18.964252 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:18.964505 24922 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:18.964298 24886 consensus_queue.cc:237] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.965111 24722 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 1.
I20260709 13:07:18.965548 24922 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.965937 24922 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.967485 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
W20260709 13:07:18.968614 24763 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.969736 24921 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:18.970144 24921 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ffac5c72add4faeadaea45e0591fb40)
I20260709 13:07:18.970525 24921 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.974282 24544 catalog_manager.cc:5697] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:18.976766 24921 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.977138 24922 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:18.979183 24921 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:18.981006 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:18.981863 24722 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ada50d39a793410cb2f6ecde5e0acbae in current term 2: Already voted for candidate 66f84d8c08484106af287bd611bc5730 in this term.
I20260709 13:07:18.983137 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:18.983856 24796 raft_consensus.cc:2393] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 66f84d8c08484106af287bd611bc5730 in current term 2: Already voted for candidate ada50d39a793410cb2f6ecde5e0acbae in this term.
W20260709 13:07:18.984268 24689 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
W20260709 13:07:18.984565 24763 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:18.984333 24922 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:18.985816 24690 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66f84d8c08484106af287bd611bc5730; no voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:18.985917 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ada50d39a793410cb2f6ecde5e0acbae; no voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:18.986666 24921 raft_consensus.cc:2749] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:18.986927 24922 raft_consensus.cc:2749] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:19.190974 24922 raft_consensus.cc:493] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.191457 24922 raft_consensus.cc:515] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:19.193241 24922 leader_election.cc:290] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:19.194444 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:19.195083 24796 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 1.
I20260709 13:07:19.196158 24690 leader_election.cc:304] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:19.196956 24922 raft_consensus.cc:2804] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:19.197371 24922 raft_consensus.cc:493] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.197743 24922 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:19.198156 24689 leader_election.cc:336] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:19.204792 24922 raft_consensus.cc:515] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:19.206681 24922 leader_election.cc:290] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:19.207886 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db90ee8362143fe9d5a6e9e91591b75" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:19.208575 24796 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:19.210142 24689 leader_election.cc:336] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:19.213543 24796 raft_consensus.cc:2468] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:19.214587 24690 leader_election.cc:304] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:19.215302 24922 raft_consensus.cc:2804] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:19.215724 24922 raft_consensus.cc:697] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:19.216486 24922 consensus_queue.cc:237] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:19.223930 24544 catalog_manager.cc:5697] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:19.317345 24689 consensus_peers.cc:597] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:19.465000 24796 raft_consensus.cc:1275] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:19.466485 24922 consensus_queue.cc:1048] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:19.472556 24689 consensus_peers.cc:597] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:19.593039 24796 raft_consensus.cc:1275] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:19.594419 24886 consensus_queue.cc:1048] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:19.708217 24796 raft_consensus.cc:1275] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:19.709565 24934 consensus_queue.cc:1048] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:19.722513 24689 consensus_peers.cc:597] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:19.779582 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:19.996897 24763 consensus_peers.cc:597] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:20.047153 24763 consensus_peers.cc:597] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:20.108218 24689 consensus_peers.cc:597] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:20.125790 24763 consensus_peers.cc:597] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:20.179169 24763 consensus_peers.cc:597] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:20.303836 24689 consensus_peers.cc:597] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:20.569195 24936 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.569656 24936 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:20.571404 24936 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:20.572981 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:20.573731 24722 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 2.
I20260709 13:07:20.574962 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
W20260709 13:07:20.575676 24763 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:20.575692 24936 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:20.576241 24936 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.576555 24936 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.581770 24936 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:20.583516 24936 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:20.585007 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:20.585698 24722 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:20.587155 24763 leader_election.cc:336] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111)
I20260709 13:07:20.590764 24722 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:20.591851 24763 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:20.592566 24936 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.593583 24936 raft_consensus.cc:697] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:20.594323 24936 consensus_queue.cc:237] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:20.602602 24547 catalog_manager.cc:5697] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 1 to 3, leader changed from 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 3 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:20.780608 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:21.080871 24763 consensus_peers.cc:597] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.131963 24722 raft_consensus.cc:1275] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:21.133302 24936 consensus_queue.cc:1048] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:21.526185 24689 consensus_peers.cc:597] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:21.781713 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:21.980759 24689 consensus_peers.cc:597] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:22.268784 24689 consensus_peers.cc:597] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:22.402551 24689 consensus_peers.cc:597] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:22.419997 24763 consensus_peers.cc:597] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:22.491988 24763 consensus_peers.cc:597] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:22.514853 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.521625 24949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.523509 24950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.523669 24952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.527393 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:22.528535 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.528825 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.529096 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602442529080 us; error 0 us; skew 500 ppm
I20260709 13:07:22.529678 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.532526 13967 webserver.cc:533] Webserver started at http://127.13.163.193:34849/ using document root <none> and password file <none>
I20260709 13:07:22.533149 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.533368 13967 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:22.537017 13967 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:22.540958 24957 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.542021 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:22.542373 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "8ffac5c72add4faeadaea45e0591fb40"
format_stamp: "Formatted at 2026-07-09 13:07:15 on dist-test-slave-w3b1"
I20260709 13:07:22.542750 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.571882 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.573184 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.574949 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.579183 24964 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:07:22.593050 24763 consensus_peers.cc:597] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:22.634521 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:07:22.634896 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.057s	user 0.001s	sys 0.002s
I20260709 13:07:22.635126 13967 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:07:22.639961 24964 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.654289 24964 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.655205 24964 tablet_bootstrap.cc:492] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.655920 24964 ts_tablet_manager.cc:1403] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:07:22.658650 24964 raft_consensus.cc:359] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.659413 24964 raft_consensus.cc:740] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.660184 24964 consensus_queue.cc:260] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.663054 24964 ts_tablet_manager.cc:1434] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
I20260709 13:07:22.663914 24964 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.678027 24964 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.678929 24964 tablet_bootstrap.cc:492] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.679702 24964 ts_tablet_manager.cc:1403] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
W20260709 13:07:22.681934 24763 consensus_peers.cc:597] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:22.682341 24964 raft_consensus.cc:359] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.683127 24964 raft_consensus.cc:385] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:22.683414 24964 raft_consensus.cc:740] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.684147 24964 consensus_queue.cc:260] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.686192 24964 ts_tablet_manager.cc:1434] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:22.686985 24964 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.696635 13967 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:07:22.697039 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.062s	user 0.059s	sys 0.003s
I20260709 13:07:22.702860 24964 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.703945 24964 tablet_bootstrap.cc:492] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.704648 24964 ts_tablet_manager.cc:1403] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.002s
I20260709 13:07:22.708369 24964 raft_consensus.cc:359] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.709478 24964 raft_consensus.cc:740] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.710589 24964 consensus_queue.cc:260] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.713632 24964 ts_tablet_manager.cc:1434] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:07:22.714705 24964 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.731678 24964 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.732563 24964 tablet_bootstrap.cc:492] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.733202 24964 ts_tablet_manager.cc:1403] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 13:07:22.735584 24964 raft_consensus.cc:359] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.736341 24964 raft_consensus.cc:740] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.737074 24964 consensus_queue.cc:260] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.739372 24964 ts_tablet_manager.cc:1434] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:22.740351 24964 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.754662 24964 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.755319 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:33301
I20260709 13:07:22.755450 25026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:33301 every 8 connection(s)
I20260709 13:07:22.755679 24964 tablet_bootstrap.cc:492] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.756610 24964 ts_tablet_manager.cc:1403] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:07:22.759526 24964 raft_consensus.cc:359] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:22.760407 24964 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.761016 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:22.761263 24964 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:22.762995 24964 ts_tablet_manager.cc:1434] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:22.763845 24964 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
W20260709 13:07:22.784665 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:22.798337 24964 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:07:22.799506 24964 tablet_bootstrap.cc:492] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.800213 24964 ts_tablet_manager.cc:1403] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 13:07:22.802214 24964 raft_consensus.cc:359] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.803135 24964 raft_consensus.cc:740] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.803951 24964 consensus_queue.cc:260] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.806298 25001 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:22.806625 24964 ts_tablet_manager.cc:1434] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:22.806491 24797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
mode: GRACEFUL
new_leader_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:53424
I20260709 13:07:22.807308 24797 raft_consensus.cc:606] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Received request to transfer leadership to TS 66f84d8c08484106af287bd611bc5730
I20260709 13:07:22.807515 24964 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.809195 25027 heartbeater.cc:344] Connected to a master server at 127.13.163.254:38747
I20260709 13:07:22.809607 25027 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.810577 25027 heartbeater.cc:507] Master 127.13.163.254:38747 requested a full tablet report, sending...
I20260709 13:07:22.825515 24547 ts_manager.cc:194] Re-registered known tserver with Master: 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301)
I20260709 13:07:22.828930 24722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
mode: GRACEFUL
new_leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:07:22.829550 24722 raft_consensus.cc:606] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Received request to transfer leadership to TS ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:22.830193 24964 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:07:22.830884 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:22.831240 24964 tablet_bootstrap.cc:492] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.831481 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:22.831979 24964 ts_tablet_manager.cc:1403] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 13:07:22.832159 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:22.834446 24964 raft_consensus.cc:359] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.835387 24964 raft_consensus.cc:740] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.836150 24964 consensus_queue.cc:260] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.838034 24547 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59640
I20260709 13:07:22.838140 24964 ts_tablet_manager.cc:1434] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:22.839176 24964 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.841595 25027 heartbeater.cc:499] Master 127.13.163.254:38747 was elected leader, sending a full tablet report...
I20260709 13:07:22.849829 24964 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.850711 24964 tablet_bootstrap.cc:492] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.851418 24964 ts_tablet_manager.cc:1403] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:07:22.853709 24964 raft_consensus.cc:359] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.854379 24964 raft_consensus.cc:740] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.855038 24964 consensus_queue.cc:260] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.857137 24964 ts_tablet_manager.cc:1434] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:07:22.857908 24964 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.869354 24964 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.870110 24964 tablet_bootstrap.cc:492] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.870877 24964 ts_tablet_manager.cc:1403] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:07:22.873611 24964 raft_consensus.cc:359] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.874516 24948 raft_consensus.cc:993] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: : Instructing follower ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:22.874537 24964 raft_consensus.cc:740] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.875098 24948 raft_consensus.cc:1081] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Signalling peer ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:22.875656 24964 consensus_queue.cc:260] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.876085 25001 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.880260 24964 ts_tablet_manager.cc:1434] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:22.881260 24964 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap starting.
I20260709 13:07:22.881587 24797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:37134
I20260709 13:07:22.882282 24797 raft_consensus.cc:493] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:22.882668 25000 raft_consensus.cc:3060] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.882643 24797 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.891899 25000 pending_rounds.cc:85] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Aborting all ops after (but not including) 0
I20260709 13:07:22.892370 25000 pending_rounds.cc:107] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:07:22.892860 25000 raft_consensus.cc:2889] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:22.897706 24797 raft_consensus.cc:515] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.900805 24797 leader_election.cc:290] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:22.902936 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:22.903615 24722 raft_consensus.cc:3055] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.903016 24999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4301a6a9cb41b88e923d3bab6f4871" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:22.904063 24722 raft_consensus.cc:740] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:22.905052 24722 consensus_queue.cc:260] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
W20260709 13:07:22.905815 24763 leader_election.cc:343] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:22.906448 24722 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.913436 24722 raft_consensus.cc:2468] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 2.
I20260709 13:07:22.913683 24964 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:22.914463 24763 leader_election.cc:304] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae; no voters: 8ffac5c72add4faeadaea45e0591fb40
I20260709 13:07:22.914592 24964 tablet_bootstrap.cc:492] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Bootstrap complete.
I20260709 13:07:22.915468 24964 ts_tablet_manager.cc:1403] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Time spent bootstrapping tablet: real 0.034s	user 0.029s	sys 0.004s
I20260709 13:07:22.915643 25033 raft_consensus.cc:2804] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:22.917538 24964 raft_consensus.cc:359] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.918259 24964 raft_consensus.cc:740] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ffac5c72add4faeadaea45e0591fb40, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.918347 25033 raft_consensus.cc:697] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 2 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:22.919019 24964 consensus_queue.cc:260] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.919377 25033 consensus_queue.cc:237] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.921382 24964 ts_tablet_manager.cc:1434] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:22.929971 24544 catalog_manager.cc:5697] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 1 to 2, leader changed from 66f84d8c08484106af287bd611bc5730 (127.13.163.194) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 2 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:22.993180 25046 raft_consensus.cc:993] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: : Instructing follower 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:22.993932 25033 raft_consensus.cc:1081] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Signalling peer 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:22.995517 24722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:49260
I20260709 13:07:22.995748 24966 raft_consensus.cc:493] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.996011 24722 raft_consensus.cc:493] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:22.996488 24722 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.996290 24966 raft_consensus.cc:515] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:22.998713 24966 leader_election.cc:290] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:23.004200 24722 raft_consensus.cc:515] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.006444 24722 leader_election.cc:290] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:23.008221 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:23.008927 25000 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.008826 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e8dbfb50f4feabc201757f9db629f" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:23.009684 24797 raft_consensus.cc:3055] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:23.010015 24797 raft_consensus.cc:740] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:23.010878 24797 consensus_queue.cc:260] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.012185 24797 raft_consensus.cc:3060] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.012918 24796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae" is_pre_election: true
I20260709 13:07:23.012945 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521301c9bcc64efb8a6f5069fedc3a05" candidate_uuid: "8ffac5c72add4faeadaea45e0591fb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f84d8c08484106af287bd611bc5730" is_pre_election: true
I20260709 13:07:23.015511 24960 leader_election.cc:304] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40; no voters: 66f84d8c08484106af287bd611bc5730, ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:23.016242 24966 raft_consensus.cc:2749] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.016788 25000 raft_consensus.cc:2468] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:23.017956 24689 leader_election.cc:304] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:23.018615 25041 raft_consensus.cc:2804] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:23.019048 25041 raft_consensus.cc:697] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:23.019164 24797 raft_consensus.cc:2468] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:23.019860 25041 consensus_queue.cc:237] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.028141 24547 catalog_manager.cc:5697] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.058473 25000 raft_consensus.cc:3060] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.083645 25000 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.091068 25000 pending_rounds.cc:85] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Aborting all ops after (but not including) 0
I20260709 13:07:23.091367 25000 pending_rounds.cc:107] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:07:23.091634 25000 raft_consensus.cc:2889] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.310149 24722 raft_consensus.cc:1275] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:23.311946 24944 consensus_queue.cc:1048] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:23.326334 25000 raft_consensus.cc:3060] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.336009 25000 raft_consensus.cc:1275] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:23.341637 25033 consensus_queue.cc:1048] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.408476 25000 raft_consensus.cc:1275] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:23.409823 25062 consensus_queue.cc:1048] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.421063 24797 raft_consensus.cc:1275] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:23.422860 25041 consensus_queue.cc:1048] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:23.786651 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:23.839933 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:23.847923 24722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
mode: GRACEFUL
new_leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:07:23.848500 24722 raft_consensus.cc:606] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Received request to transfer leadership to TS ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:23.851069 24797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
mode: GRACEFUL
new_leader_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:53424
I20260709 13:07:23.851577 24797 raft_consensus.cc:606] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Received request to transfer leadership to TS 66f84d8c08484106af287bd611bc5730
I20260709 13:07:23.852780 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:23.853649 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:23.854374 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:23.874585 24948 raft_consensus.cc:993] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: : Instructing follower ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:23.875149 25041 raft_consensus.cc:1081] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Signalling peer ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:23.876760 24797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:37134
I20260709 13:07:23.877316 24797 raft_consensus.cc:493] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:23.877633 24797 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.882509 24797 raft_consensus.cc:515] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.884109 24797 leader_election.cc:290] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:23.885125 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:23.885367 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce8ce5e5cc44212a8163ba44d266611" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:23.885895 25000 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.886000 24722 raft_consensus.cc:3055] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:23.886445 24722 raft_consensus.cc:740] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:23.887168 24722 consensus_queue.cc:260] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.888108 24722 raft_consensus.cc:3060] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.893106 25000 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:23.894546 24763 leader_election.cc:304] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:23.894645 24722 raft_consensus.cc:2468] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:23.895337 25033 raft_consensus.cc:2804] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:23.895831 25033 raft_consensus.cc:697] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:23.896618 25033 consensus_queue.cc:237] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:23.903877 24547 catalog_manager.cc:5697] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 2 to 3, leader changed from 66f84d8c08484106af287bd611bc5730 (127.13.163.194) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 3 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.949568 24944 raft_consensus.cc:993] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: : Instructing follower 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:23.950043 25033 raft_consensus.cc:1081] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Signalling peer 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:23.951383 24722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:49260
I20260709 13:07:23.951840 24722 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:23.952128 24722 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.956650 24722 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:23.958166 24722 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769), 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301)
I20260709 13:07:23.959259 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:23.959141 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:23.960008 25000 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.960057 24797 raft_consensus.cc:3055] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:23.960556 24797 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:23.961490 24797 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:23.962927 24797 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.968264 25000 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:23.969790 24689 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:23.970650 25041 raft_consensus.cc:2804] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:23.971024 24797 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:23.971079 25041 raft_consensus.cc:697] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:23.972270 25041 consensus_queue.cc:237] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:23.978998 24547 catalog_manager.cc:5697] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:24.301010 25000 raft_consensus.cc:1275] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:24.302235 25033 consensus_queue.cc:1048] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.312143 24722 raft_consensus.cc:1275] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:24.314043 25033 consensus_queue.cc:1048] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.465409 25000 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.467937 24948 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.488637 24797 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.490414 24948 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:24.787855 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:24.861717 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:24.873512 24722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
mode: GRACEFUL
new_leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:07:24.874138 24722 raft_consensus.cc:606] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Received request to transfer leadership to TS ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:24.877225 24797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
mode: GRACEFUL
new_leader_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:53424
I20260709 13:07:24.877781 24797 raft_consensus.cc:606] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Received request to transfer leadership to TS 66f84d8c08484106af287bd611bc5730
I20260709 13:07:24.879338 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:24.879885 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:24.880631 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:25.006207 24948 raft_consensus.cc:993] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: : Instructing follower ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:25.006712 24948 raft_consensus.cc:1081] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Signalling peer ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:25.008349 24797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:37134
I20260709 13:07:25.009056 24797 raft_consensus.cc:493] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.009425 24797 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.018934 24797 raft_consensus.cc:515] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.020639 24797 leader_election.cc:290] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:25.021931 25001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:25.022042 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1571d44d0df046f6b2db3f60b400d192" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:25.022658 25001 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.022670 24722 raft_consensus.cc:3055] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:25.023145 24722 raft_consensus.cc:740] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:25.024084 24722 consensus_queue.cc:260] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.025239 24722 raft_consensus.cc:3060] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.029893 25001 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:25.031427 24763 leader_election.cc:304] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:25.031865 24722 raft_consensus.cc:2468] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:25.032289 25081 raft_consensus.cc:2804] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:25.032876 25081 raft_consensus.cc:697] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:25.033957 25081 consensus_queue.cc:237] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.042181 24545 catalog_manager.cc:5697] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 2 to 3, leader changed from 66f84d8c08484106af287bd611bc5730 (127.13.163.194) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 3 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.208671 25033 raft_consensus.cc:993] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: : Instructing follower 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:25.209215 25081 raft_consensus.cc:1081] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Signalling peer 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:25.210796 24722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:49260
I20260709 13:07:25.211373 24722 raft_consensus.cc:493] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.211762 24722 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.217182 24722 raft_consensus.cc:515] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.218895 24722 leader_election.cc:290] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:25.220039 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:25.219857 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2bcf9425744c3c96f9ba223fa12da3" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:25.220625 24797 raft_consensus.cc:3055] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.220880 25000 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.221012 24797 raft_consensus.cc:740] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:25.221927 24797 consensus_queue.cc:260] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.222977 24797 raft_consensus.cc:3060] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.227741 25000 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:25.227988 24797 raft_consensus.cc:2468] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 2.
I20260709 13:07:25.229260 24689 leader_election.cc:304] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:25.230026 24948 raft_consensus.cc:2804] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.230505 24948 raft_consensus.cc:697] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:25.231423 24948 consensus_queue.cc:237] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:25.239403 24545 catalog_manager.cc:5697] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 1 to 2, leader changed from ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:25.496083 24722 raft_consensus.cc:1275] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:25.497437 25081 consensus_queue.cc:1048] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:25.508723 25000 raft_consensus.cc:1275] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:25.510241 25081 consensus_queue.cc:1048] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:25.611415 24797 raft_consensus.cc:1275] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.612910 24948 consensus_queue.cc:1048] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.623394 25000 raft_consensus.cc:1275] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.625735 25095 consensus_queue.cc:1048] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:25.788969 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:25.886768 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:25.894186 24722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
mode: GRACEFUL
new_leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:07:25.894840 24722 raft_consensus.cc:606] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Received request to transfer leadership to TS ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:25.897465 24797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
mode: GRACEFUL
new_leader_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:53424
I20260709 13:07:25.897897 24797 raft_consensus.cc:606] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Received request to transfer leadership to TS 66f84d8c08484106af287bd611bc5730
I20260709 13:07:25.899093 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:25.899574 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:25.900166 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:25.978475 25095 raft_consensus.cc:993] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: : Instructing follower ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:25.979147 25041 raft_consensus.cc:1081] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Signalling peer ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:25.980674 24797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:37134
I20260709 13:07:25.981235 24797 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.981597 24797 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.987040 24797 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:25.988840 24797 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:25.989774 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:25.989663 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:25.990489 24722 raft_consensus.cc:3055] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:25.990545 25000 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.990800 24722 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:25.991734 24722 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:25.992719 24722 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.996270 25000 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:25.997530 24763 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:25.997923 24722 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 3.
I20260709 13:07:25.998446 25033 raft_consensus.cc:2804] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:25.998971 25033 raft_consensus.cc:697] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:25.999893 25033 consensus_queue.cc:237] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:26.008070 24545 catalog_manager.cc:5697] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 2 to 3, leader changed from 66f84d8c08484106af287bd611bc5730 (127.13.163.194) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 3 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:26.150935 25101 raft_consensus.cc:993] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: : Instructing follower 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:26.151449 25033 raft_consensus.cc:1081] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Signalling peer 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:26.152922 24722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:49260
I20260709 13:07:26.153410 24722 raft_consensus.cc:493] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:26.153780 24722 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.158836 24722 raft_consensus.cc:515] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:26.160532 24722 leader_election.cc:290] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 4 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769)
I20260709 13:07:26.161492 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:26.162073 25000 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.162341 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beeeceddef6941b88cc25e1a802568f1" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:26.162948 24797 raft_consensus.cc:3055] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:26.163261 24797 raft_consensus.cc:740] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:26.164006 24797 consensus_queue.cc:260] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:26.165048 24797 raft_consensus.cc:3060] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.167184 25000 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 4.
I20260709 13:07:26.168272 24689 leader_election.cc:304] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:26.168973 25041 raft_consensus.cc:2804] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:26.169420 25041 raft_consensus.cc:697] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 4 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:26.170117 24797 raft_consensus.cc:2468] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 4.
I20260709 13:07:26.170499 25041 consensus_queue.cc:237] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:26.179509 24545 catalog_manager.cc:5697] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 3 to 4, leader changed from ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 4 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:26.516431 25000 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:26.517690 25033 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:26.527051 24722 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:26.529131 25101 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:26.627867 24797 raft_consensus.cc:1275] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:26.629053 25041 consensus_queue.cc:1048] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:26.639214 25000 raft_consensus.cc:1275] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:26.640681 25110 consensus_queue.cc:1048] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 13:07:26.790107 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:26.906395 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:26.913923 24722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
mode: GRACEFUL
new_leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:07:26.914469 24722 raft_consensus.cc:606] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Received request to transfer leadership to TS ada50d39a793410cb2f6ecde5e0acbae
I20260709 13:07:26.917224 24797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
mode: GRACEFUL
new_leader_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:53424
I20260709 13:07:26.917640 24797 raft_consensus.cc:606] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Received request to transfer leadership to TS 66f84d8c08484106af287bd611bc5730
I20260709 13:07:26.918771 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:26.919276 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:26.919855 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.011354 25101 raft_consensus.cc:993] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: : Instructing follower 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:27.011929 25081 raft_consensus.cc:1081] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Signalling peer 66f84d8c08484106af287bd611bc5730 to start an election
I20260709 13:07:27.013561 24722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491"
dest_uuid: "66f84d8c08484106af287bd611bc5730"
 from {username='slave'} at 127.0.0.1:49260
I20260709 13:07:27.014030 24722 raft_consensus.cc:493] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.014356 24722 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:27.019259 24722 raft_consensus.cc:515] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:27.020931 24722 leader_election.cc:290] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 4 election: Requested vote from peers ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195:37769), 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301)
I20260709 13:07:27.021955 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:27.022253 24797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b6ed18ec1444c899ca33345411f491" candidate_uuid: "66f84d8c08484106af287bd611bc5730" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
I20260709 13:07:27.022523 25000 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:27.022863 24797 raft_consensus.cc:3055] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:27.023155 24797 raft_consensus.cc:740] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:27.023861 24797 consensus_queue.cc:260] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:27.025010 24797 raft_consensus.cc:3060] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:27.028868 25000 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 4.
I20260709 13:07:27.029919 24689 leader_election.cc:304] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f84d8c08484106af287bd611bc5730, 8ffac5c72add4faeadaea45e0591fb40; no voters: 
I20260709 13:07:27.030623 25095 raft_consensus.cc:2804] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:27.031121 25095 raft_consensus.cc:697] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 4 LEADER]: Becoming Leader. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:27.031488 24797 raft_consensus.cc:2468] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66f84d8c08484106af287bd611bc5730 in term 4.
I20260709 13:07:27.032038 25095 consensus_queue.cc:237] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } }
I20260709 13:07:27.038203 24545 catalog_manager.cc:5697] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 reported cstate change: term changed from 3 to 4, leader changed from ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195) to 66f84d8c08484106af287bd611bc5730 (127.13.163.194). New cstate: current_term: 4 leader_uuid: "66f84d8c08484106af287bd611bc5730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.068605 25110 raft_consensus.cc:993] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: : Instructing follower ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:27.069099 25095 raft_consensus.cc:1081] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Signalling peer ada50d39a793410cb2f6ecde5e0acbae to start an election
I20260709 13:07:27.070675 24797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6"
dest_uuid: "ada50d39a793410cb2f6ecde5e0acbae"
 from {username='slave'} at 127.0.0.1:37134
I20260709 13:07:27.071074 24797 raft_consensus.cc:493] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.071302 24797 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.076232 24797 raft_consensus.cc:515] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:27.078133 24797 leader_election.cc:290] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Requested vote from peers 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301), 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851)
I20260709 13:07:27.078953 25000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ffac5c72add4faeadaea45e0591fb40"
I20260709 13:07:27.079704 25000 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.080060 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1638e127d7944b8a9675e92811ac8db6" candidate_uuid: "ada50d39a793410cb2f6ecde5e0acbae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66f84d8c08484106af287bd611bc5730"
I20260709 13:07:27.080668 24722 raft_consensus.cc:3055] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.080960 24722 raft_consensus.cc:740] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66f84d8c08484106af287bd611bc5730, State: Running, Role: LEADER
I20260709 13:07:27.081880 24722 consensus_queue.cc:260] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:27.083189 24722 raft_consensus.cc:3060] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.086848 25000 raft_consensus.cc:2468] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 2.
I20260709 13:07:27.088111 24763 leader_election.cc:304] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ffac5c72add4faeadaea45e0591fb40, ada50d39a793410cb2f6ecde5e0acbae; no voters: 
I20260709 13:07:27.089015 25081 raft_consensus.cc:2804] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.089573 25081 raft_consensus.cc:697] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 2 LEADER]: Becoming Leader. State: Replica: ada50d39a793410cb2f6ecde5e0acbae, State: Running, Role: LEADER
I20260709 13:07:27.090600 24722 raft_consensus.cc:2468] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada50d39a793410cb2f6ecde5e0acbae in term 2.
I20260709 13:07:27.091293 25081 consensus_queue.cc:237] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } }
I20260709 13:07:27.099610 24545 catalog_manager.cc:5697] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae reported cstate change: term changed from 1 to 2, leader changed from 66f84d8c08484106af287bd611bc5730 (127.13.163.194) to ada50d39a793410cb2f6ecde5e0acbae (127.13.163.195). New cstate: current_term: 2 leader_uuid: "ada50d39a793410cb2f6ecde5e0acbae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.520625 24797 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:27.521921 25116 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada50d39a793410cb2f6ecde5e0acbae" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:27.531360 25000 raft_consensus.cc:1275] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Refusing update from remote peer 66f84d8c08484106af287bd611bc5730: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:27.532780 25117 consensus_queue.cc:1048] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:27.566273 25000 raft_consensus.cc:1275] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.567643 25101 consensus_queue.cc:1048] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ffac5c72add4faeadaea45e0591fb40" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:27.577832 24722 raft_consensus.cc:1275] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Refusing update from remote peer ada50d39a793410cb2f6ecde5e0acbae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.580509 25124 consensus_queue.cc:1048] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f84d8c08484106af287bd611bc5730" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:27.791085 24595 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:27.925997 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.954833 13967 tablet_server.cc:179] TabletServer@127.13.163.193:33301 shutting down...
W20260709 13:07:27.982389 24689 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 86 similar messages]
W20260709 13:07:27.988281 24689 consensus_peers.cc:597] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:27.989948 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:27.990736 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:27.992149 13967 raft_consensus.cc:2243] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:27.992892 13967 raft_consensus.cc:2272] T dd4301a6a9cb41b88e923d3bab6f4871 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:27.995203 24689 consensus_peers.cc:597] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:27.996479 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:27.997448 13967 raft_consensus.cc:2243] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:27.998149 13967 raft_consensus.cc:2272] T fc2bcf9425744c3c96f9ba223fa12da3 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.000576 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.001232 13967 raft_consensus.cc:2243] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.001760 13967 raft_consensus.cc:2272] T 0ce8ce5e5cc44212a8163ba44d266611 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.004190 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.004797 13967 raft_consensus.cc:2243] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.005352 13967 raft_consensus.cc:2272] T 5db90ee8362143fe9d5a6e9e91591b75 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.007838 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.008425 13967 raft_consensus.cc:2243] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.008966 13967 raft_consensus.cc:2272] T b15e8dbfb50f4feabc201757f9db629f P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.011339 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.012032 13967 raft_consensus.cc:2243] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.012472 13967 raft_consensus.cc:2272] T 1638e127d7944b8a9675e92811ac8db6 P 8ffac5c72add4faeadaea45e0591fb40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.016485 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.017191 13967 raft_consensus.cc:2243] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.017686 13967 raft_consensus.cc:2272] T 1571d44d0df046f6b2db3f60b400d192 P 8ffac5c72add4faeadaea45e0591fb40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.019968 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.020495 13967 raft_consensus.cc:2243] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.020985 13967 raft_consensus.cc:2272] T 25b6ed18ec1444c899ca33345411f491 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.023751 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.024307 13967 raft_consensus.cc:2243] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.024761 13967 raft_consensus.cc:2272] T beeeceddef6941b88cc25e1a802568f1 P 8ffac5c72add4faeadaea45e0591fb40 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.026985 13967 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:28.027458 13967 raft_consensus.cc:2243] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.027940 13967 raft_consensus.cc:2272] T 521301c9bcc64efb8a6f5069fedc3a05 P 8ffac5c72add4faeadaea45e0591fb40 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:28.040215 24689 consensus_peers.cc:597] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.063486 13967 tablet_server.cc:196] TabletServer@127.13.163.193:33301 shutdown complete.
W20260709 13:07:28.064167 24763 consensus_peers.cc:597] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.093516 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
W20260709 13:07:28.103149 24763 consensus_peers.cc:597] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Couldn't send request to peer 66f84d8c08484106af287bd611bc5730. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:28.124261 24689 consensus_peers.cc:597] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.126176 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:28.126878 13967 tablet_replica.cc:333] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.127629 13967 raft_consensus.cc:2243] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.128423 13967 raft_consensus.cc:2272] T fc2bcf9425744c3c96f9ba223fa12da3 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.130730 13967 tablet_replica.cc:333] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.131384 13967 raft_consensus.cc:2243] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.132269 13967 raft_consensus.cc:2272] T 5db90ee8362143fe9d5a6e9e91591b75 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.134711 13967 tablet_replica.cc:333] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.135275 13967 raft_consensus.cc:2243] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.135775 13967 raft_consensus.cc:2272] T dd4301a6a9cb41b88e923d3bab6f4871 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.138003 13967 tablet_replica.cc:333] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.138507 13967 raft_consensus.cc:2243] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.139016 13967 raft_consensus.cc:2272] T 0ce8ce5e5cc44212a8163ba44d266611 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.141081 13967 tablet_replica.cc:333] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.141614 13967 raft_consensus.cc:2243] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.142103 13967 raft_consensus.cc:2272] T 521301c9bcc64efb8a6f5069fedc3a05 P 66f84d8c08484106af287bd611bc5730 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.144196 13967 tablet_replica.cc:333] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.144769 13967 raft_consensus.cc:2243] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.145236 13967 raft_consensus.cc:2272] T 1571d44d0df046f6b2db3f60b400d192 P 66f84d8c08484106af287bd611bc5730 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.147428 13967 tablet_replica.cc:333] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.147976 13967 raft_consensus.cc:2243] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.148797 13967 raft_consensus.cc:2272] T b15e8dbfb50f4feabc201757f9db629f P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.150992 13967 tablet_replica.cc:333] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.151522 13967 raft_consensus.cc:2243] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.151916 13967 raft_consensus.cc:2272] T 1638e127d7944b8a9675e92811ac8db6 P 66f84d8c08484106af287bd611bc5730 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.154462 13967 tablet_replica.cc:333] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.155006 13967 raft_consensus.cc:2243] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.155807 13967 raft_consensus.cc:2272] T 25b6ed18ec1444c899ca33345411f491 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.158819 13967 tablet_replica.cc:333] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730: stopping tablet replica
I20260709 13:07:28.159503 13967 raft_consensus.cc:2243] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.160449 13967 raft_consensus.cc:2272] T beeeceddef6941b88cc25e1a802568f1 P 66f84d8c08484106af287bd611bc5730 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:28.161828 24763 consensus_peers.cc:597] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:28.176146 24763 consensus_peers.cc:597] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Couldn't send request to peer 66f84d8c08484106af287bd611bc5730. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:28.178812 24763 consensus_peers.cc:597] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:28.181332 24689 connection.cc:466] server connection from 127.0.0.1:49260 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49260 (request call id 149) could send its response
W20260709 13:07:28.182279 24763 consensus_peers.cc:597] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Couldn't send request to peer 66f84d8c08484106af287bd611bc5730. Status: Network error: recv got EOF from 127.13.163.194:32851 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.189745 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
I20260709 13:07:28.216522 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
W20260709 13:07:28.224088 24763 consensus_peers.cc:597] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 66f84d8c08484106af287bd611bc5730 (127.13.163.194:32851): Couldn't send request to peer 66f84d8c08484106af287bd611bc5730. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:32851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.247007 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:07:28.247293 24763 consensus_peers.cc:597] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae -> Peer 8ffac5c72add4faeadaea45e0591fb40 (127.13.163.193:33301): Couldn't send request to peer 8ffac5c72add4faeadaea45e0591fb40. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:33301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:28.247952 13967 tablet_replica.cc:333] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.249058 13967 raft_consensus.cc:2243] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.249835 13967 raft_consensus.cc:2272] T 0ce8ce5e5cc44212a8163ba44d266611 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.251928 13967 tablet_replica.cc:333] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.252434 13967 raft_consensus.cc:2243] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.252912 13967 raft_consensus.cc:2272] T 5db90ee8362143fe9d5a6e9e91591b75 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.254927 13967 tablet_replica.cc:333] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.255373 13967 raft_consensus.cc:2243] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.256150 13967 raft_consensus.cc:2272] T 521301c9bcc64efb8a6f5069fedc3a05 P ada50d39a793410cb2f6ecde5e0acbae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.258361 13967 tablet_replica.cc:333] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.258852 13967 raft_consensus.cc:2243] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.259204 13967 raft_consensus.cc:2272] T fc2bcf9425744c3c96f9ba223fa12da3 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.261201 13967 tablet_replica.cc:333] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.261751 13967 raft_consensus.cc:2243] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.262439 13967 raft_consensus.cc:2272] T 1638e127d7944b8a9675e92811ac8db6 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.265450 13967 tablet_replica.cc:333] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.265955 13967 raft_consensus.cc:2243] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.266630 13967 raft_consensus.cc:2272] T 1571d44d0df046f6b2db3f60b400d192 P ada50d39a793410cb2f6ecde5e0acbae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.268684 13967 tablet_replica.cc:333] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.269235 13967 raft_consensus.cc:2243] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.269721 13967 raft_consensus.cc:2272] T b15e8dbfb50f4feabc201757f9db629f P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.271649 13967 tablet_replica.cc:333] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.272094 13967 raft_consensus.cc:2243] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.273054 13967 raft_consensus.cc:2272] T dd4301a6a9cb41b88e923d3bab6f4871 P ada50d39a793410cb2f6ecde5e0acbae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.275182 13967 tablet_replica.cc:333] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.275696 13967 raft_consensus.cc:2243] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.276041 13967 raft_consensus.cc:2272] T 25b6ed18ec1444c899ca33345411f491 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.278448 13967 tablet_replica.cc:333] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae: stopping tablet replica
I20260709 13:07:28.278954 13967 raft_consensus.cc:2243] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:28.279297 13967 raft_consensus.cc:2272] T beeeceddef6941b88cc25e1a802568f1 P ada50d39a793410cb2f6ecde5e0acbae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.311123 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:07:28.337195 13967 master.cc:562] Master@127.13.163.254:38747 shutting down...
I20260709 13:07:28.361912 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:28.362514 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:28.362881 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 53d4731a4fb54603a7c2f222c83c6028: stopping tablet replica
I20260709 13:07:28.385051 13967 master.cc:584] Master@127.13.163.254:38747 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12817 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:07:28.436514 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:39159
I20260709 13:07:28.437770 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:28.443596 25135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.443693 25136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.444327 25138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:28.445376 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:28.446277 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:28.446450 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:28.446625 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602448446611 us; error 0 us; skew 500 ppm
I20260709 13:07:28.447175 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:28.449910 13967 webserver.cc:533] Webserver started at http://127.13.163.254:46691/ using document root <none> and password file <none>
I20260709 13:07:28.450429 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:28.450620 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:28.450960 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:28.452239 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "942cc6adfc4a41e0bdc144b4001e1eb6"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.456979 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:07:28.460513 25143 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.461376 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:28.461671 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "942cc6adfc4a41e0bdc144b4001e1eb6"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.461972 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:28.486553 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:28.487874 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:28.528635 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:39159
I20260709 13:07:28.528823 25194 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:39159 every 8 connection(s)
I20260709 13:07:28.532727 25195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:28.544346 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6: Bootstrap starting.
I20260709 13:07:28.549757 25195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:28.554746 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6: No bootstrap required, opened a new log
I20260709 13:07:28.557073 25195 raft_consensus.cc:359] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER }
I20260709 13:07:28.557535 25195 raft_consensus.cc:385] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:28.557803 25195 raft_consensus.cc:740] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942cc6adfc4a41e0bdc144b4001e1eb6, State: Initialized, Role: FOLLOWER
I20260709 13:07:28.558516 25195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER }
I20260709 13:07:28.558995 25195 raft_consensus.cc:399] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:28.559230 25195 raft_consensus.cc:493] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:28.559505 25195 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:28.564648 25195 raft_consensus.cc:515] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER }
I20260709 13:07:28.565210 25195 leader_election.cc:304] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 942cc6adfc4a41e0bdc144b4001e1eb6; no voters: 
I20260709 13:07:28.566385 25195 leader_election.cc:290] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:28.566800 25198 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:28.568222 25198 raft_consensus.cc:697] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 1 LEADER]: Becoming Leader. State: Replica: 942cc6adfc4a41e0bdc144b4001e1eb6, State: Running, Role: LEADER
I20260709 13:07:28.569242 25198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER }
I20260709 13:07:28.569972 25195 sys_catalog.cc:565] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:28.574693 25200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 942cc6adfc4a41e0bdc144b4001e1eb6. Latest consensus state: current_term: 1 leader_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER } }
I20260709 13:07:28.575623 25200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:28.575683 25199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942cc6adfc4a41e0bdc144b4001e1eb6" member_type: VOTER } }
I20260709 13:07:28.576395 25199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:28.577796 25204 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:28.582795 25204 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:28.588536 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:28.592163 25204 catalog_manager.cc:1383] Generated new cluster ID: dfa23b6d986a420caf799608cd285a5f
I20260709 13:07:28.592471 25204 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:28.612740 25204 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:28.614106 25204 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:28.628479 25204 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6: Generated new TSK 0
I20260709 13:07:28.629143 25204 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:28.655843 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:28.662279 25216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.663601 25217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.665516 25219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:28.666267 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:28.667238 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:28.667459 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:28.667737 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602448667650 us; error 0 us; skew 500 ppm
I20260709 13:07:28.668383 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:28.670935 13967 webserver.cc:533] Webserver started at http://127.13.163.193:34875/ using document root <none> and password file <none>
I20260709 13:07:28.671464 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:28.671715 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:28.671993 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:28.673183 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "7f0860048032459185b00d922c323bc5"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.677933 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:07:28.681967 25224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.682955 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:28.683292 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "7f0860048032459185b00d922c323bc5"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.683605 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:28.712085 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:28.713424 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:28.714996 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:28.717437 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:28.717682 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.717947 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:28.718138 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.761145 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:35739
I20260709 13:07:28.761261 25286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:35739 every 8 connection(s)
I20260709 13:07:28.768157 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:28.776592 25291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.780390 25292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.784811 25294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:28.785012 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:28.786167 25287 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39159
I20260709 13:07:28.786370 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:28.786638 25287 heartbeater.cc:461] Registering TS with master...
W20260709 13:07:28.786760 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:28.787091 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602448787075 us; error 0 us; skew 500 ppm
I20260709 13:07:28.787662 25287 heartbeater.cc:507] Master 127.13.163.254:39159 requested a full tablet report, sending...
I20260709 13:07:28.787771 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:28.790207 25160 ts_manager.cc:194] Registered new tserver with Master: 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739)
I20260709 13:07:28.790858 13967 webserver.cc:533] Webserver started at http://127.13.163.194:43159/ using document root <none> and password file <none>
I20260709 13:07:28.791458 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:28.791683 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:28.791944 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:28.792464 25160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43638
I20260709 13:07:28.793370 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.798275 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:07:28.802024 25299 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.803056 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:28.803408 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.803731 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:28.823827 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:28.825222 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:28.826792 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:28.829118 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:28.829295 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.829593 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:28.829751 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.888470 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:34311
I20260709 13:07:28.888676 25361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:34311 every 8 connection(s)
I20260709 13:07:28.893828 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:28.901237 25365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:28.901499 25366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:28.911909 25362 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39159
I20260709 13:07:28.912457 25362 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:28.913378 13967 server_base.cc:1061] running on GCE node
W20260709 13:07:28.913465 25368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:28.913601 25362 heartbeater.cc:507] Master 127.13.163.254:39159 requested a full tablet report, sending...
I20260709 13:07:28.914902 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:28.915213 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:28.915470 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602448915434 us; error 0 us; skew 500 ppm
I20260709 13:07:28.916144 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:28.916512 25160 ts_manager.cc:194] Registered new tserver with Master: cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:28.918200 25160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43648
I20260709 13:07:28.918944 13967 webserver.cc:533] Webserver started at http://127.13.163.195:44253/ using document root <none> and password file <none>
I20260709 13:07:28.919654 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:28.919888 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:28.920171 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:28.921500 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "e1017bddda0847b2867904967a83ef54"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.926132 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:07:28.929553 25373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.930369 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:07:28.930723 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "e1017bddda0847b2867904967a83ef54"
format_stamp: "Formatted at 2026-07-09 13:07:28 on dist-test-slave-w3b1"
I20260709 13:07:28.931047 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:28.945207 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:28.946441 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:28.947908 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:28.950270 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:28.950505 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.950754 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:28.950927 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:28.993377 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:36853
I20260709 13:07:28.993479 25435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:36853 every 8 connection(s)
I20260709 13:07:29.008409 25436 heartbeater.cc:344] Connected to a master server at 127.13.163.254:39159
I20260709 13:07:29.008879 25436 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:29.009667 25436 heartbeater.cc:507] Master 127.13.163.254:39159 requested a full tablet report, sending...
I20260709 13:07:29.011608 25160 ts_manager.cc:194] Registered new tserver with Master: e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853)
I20260709 13:07:29.011700 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015066977s
I20260709 13:07:29.013128 25160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43658
I20260709 13:07:29.037559 25160 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43674:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:29.039729 25160 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:29.100217 25251 tablet_service.cc:1511] Processing CreateTablet for tablet a33408c2a79749429762468f4f930b0e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.100593 25252 tablet_service.cc:1511] Processing CreateTablet for tablet 7a88bbda569547a987b7da77d01b49cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.101764 25251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a33408c2a79749429762468f4f930b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.102829 25400 tablet_service.cc:1511] Processing CreateTablet for tablet a33408c2a79749429762468f4f930b0e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.103296 25401 tablet_service.cc:1511] Processing CreateTablet for tablet 7a88bbda569547a987b7da77d01b49cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.104372 25400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a33408c2a79749429762468f4f930b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.104800 25252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a88bbda569547a987b7da77d01b49cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.105239 25401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a88bbda569547a987b7da77d01b49cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.124331 25451 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: Bootstrap starting.
I20260709 13:07:29.131636 25452 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: Bootstrap starting.
I20260709 13:07:29.132871 25327 tablet_service.cc:1511] Processing CreateTablet for tablet 7a88bbda569547a987b7da77d01b49cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.133680 25451 tablet_bootstrap.cc:654] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.133476 25326 tablet_service.cc:1511] Processing CreateTablet for tablet a33408c2a79749429762468f4f930b0e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3aed5fa538a64b90ad5ebecc30738205]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.134239 25327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a88bbda569547a987b7da77d01b49cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.135178 25326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a33408c2a79749429762468f4f930b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.142300 25452 tablet_bootstrap.cc:654] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.157549 25452 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: No bootstrap required, opened a new log
I20260709 13:07:29.158352 25452 ts_tablet_manager.cc:1403] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 13:07:29.159754 25451 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: No bootstrap required, opened a new log
I20260709 13:07:29.160373 25451 ts_tablet_manager.cc:1403] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.016s
I20260709 13:07:29.162138 25452 raft_consensus.cc:359] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.162649 25455 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: Bootstrap starting.
I20260709 13:07:29.163079 25452 raft_consensus.cc:385] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.163479 25452 raft_consensus.cc:740] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.163931 25451 raft_consensus.cc:359] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.164354 25452 consensus_queue.cc:260] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.164959 25451 raft_consensus.cc:385] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.165491 25451 raft_consensus.cc:740] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0860048032459185b00d922c323bc5, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.169390 25451 consensus_queue.cc:260] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.173776 25436 heartbeater.cc:499] Master 127.13.163.254:39159 was elected leader, sending a full tablet report...
I20260709 13:07:29.174216 25452 ts_tablet_manager.cc:1434] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:07:29.174507 25287 heartbeater.cc:499] Master 127.13.163.254:39159 was elected leader, sending a full tablet report...
I20260709 13:07:29.175302 25452 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: Bootstrap starting.
I20260709 13:07:29.175371 25451 ts_tablet_manager.cc:1434] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 13:07:29.176697 25451 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: Bootstrap starting.
I20260709 13:07:29.173079 25455 tablet_bootstrap.cc:654] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.183781 25452 tablet_bootstrap.cc:654] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.186303 25451 tablet_bootstrap.cc:654] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.189561 25456 raft_consensus.cc:493] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.190320 25456 raft_consensus.cc:515] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.213234 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5" is_pre_election: true
I20260709 13:07:29.214263 25262 raft_consensus.cc:2468] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 0.
I20260709 13:07:29.216357 25374 leader_election.cc:304] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:29.231226 25455 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: No bootstrap required, opened a new log
I20260709 13:07:29.233650 25455 ts_tablet_manager.cc:1403] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent bootstrapping tablet: real 0.071s	user 0.019s	sys 0.004s
I20260709 13:07:29.237011 25451 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: No bootstrap required, opened a new log
I20260709 13:07:29.237632 25451 ts_tablet_manager.cc:1403] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.034s
I20260709 13:07:29.238112 25452 tablet_bootstrap.cc:492] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: No bootstrap required, opened a new log
I20260709 13:07:29.234077 25462 raft_consensus.cc:2804] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:29.238787 25452 ts_tablet_manager.cc:1403] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: Time spent bootstrapping tablet: real 0.064s	user 0.035s	sys 0.026s
I20260709 13:07:29.238997 25462 raft_consensus.cc:493] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.239595 25462 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.240458 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" is_pre_election: true
I20260709 13:07:29.241009 25451 raft_consensus.cc:359] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.241849 25451 raft_consensus.cc:385] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.242177 25451 raft_consensus.cc:740] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0860048032459185b00d922c323bc5, State: Initialized, Role: FOLLOWER
W20260709 13:07:29.242257 25375 leader_election.cc:343] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:29.242998 25452 raft_consensus.cc:359] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.243085 25451 consensus_queue.cc:260] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.238727 25455 raft_consensus.cc:359] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.244096 25452 raft_consensus.cc:385] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.244592 25452 raft_consensus.cc:740] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.245536 25452 consensus_queue.cc:260] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.234947 25456 leader_election.cc:290] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.244175 25455 raft_consensus.cc:385] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.248989 25455 raft_consensus.cc:740] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ed7972b524fa4a30b9c64e8b123c4, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.250191 25451 ts_tablet_manager.cc:1434] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:07:29.250041 25455 consensus_queue.cc:260] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.253111 25362 heartbeater.cc:499] Master 127.13.163.254:39159 was elected leader, sending a full tablet report...
I20260709 13:07:29.254617 25455 ts_tablet_manager.cc:1434] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:07:29.261080 25462 raft_consensus.cc:515] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.263615 25455 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: Bootstrap starting.
I20260709 13:07:29.264628 25462 leader_election.cc:290] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Requested vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.265525 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5"
I20260709 13:07:29.266826 25262 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.267167 25452 ts_tablet_manager.cc:1434] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.020s
I20260709 13:07:29.267343 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
W20260709 13:07:29.270216 25375 leader_election.cc:343] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:29.275163 25262 raft_consensus.cc:2468] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 1.
I20260709 13:07:29.276099 25455 tablet_bootstrap.cc:654] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.276389 25374 leader_election.cc:304] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f0860048032459185b00d922c323bc5, e1017bddda0847b2867904967a83ef54; no voters: cf9ed7972b524fa4a30b9c64e8b123c4
I20260709 13:07:29.277334 25462 raft_consensus.cc:2804] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:29.277925 25462 raft_consensus.cc:697] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Leader. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:29.278916 25462 consensus_queue.cc:237] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.282966 25455 tablet_bootstrap.cc:492] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: No bootstrap required, opened a new log
I20260709 13:07:29.283695 25455 ts_tablet_manager.cc:1403] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.004s
I20260709 13:07:29.286978 25455 raft_consensus.cc:359] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.287698 25455 raft_consensus.cc:385] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.287957 25455 raft_consensus.cc:740] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ed7972b524fa4a30b9c64e8b123c4, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.288784 25455 consensus_queue.cc:260] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.288895 25158 catalog_manager.cc:5697] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1017bddda0847b2867904967a83ef54 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "e1017bddda0847b2867904967a83ef54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.290813 25455 ts_tablet_manager.cc:1434] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:07:29.353876 25462 raft_consensus.cc:493] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.354440 25462 raft_consensus.cc:515] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.356292 25462 leader_election.cc:290] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.357384 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" is_pre_election: true
I20260709 13:07:29.357384 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5" is_pre_election: true
I20260709 13:07:29.358311 25337 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 0.
I20260709 13:07:29.358342 25262 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 0.
I20260709 13:07:29.359637 25375 leader_election.cc:304] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf9ed7972b524fa4a30b9c64e8b123c4, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:29.360405 25462 raft_consensus.cc:2804] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:29.360814 25462 raft_consensus.cc:493] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.361171 25462 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.366888 25462 raft_consensus.cc:515] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.368523 25462 leader_election.cc:290] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Requested vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.369535 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5"
I20260709 13:07:29.369626 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:29.370270 25337 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.370285 25262 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.375880 25337 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 1.
I20260709 13:07:29.376044 25262 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 1.
I20260709 13:07:29.377142 25375 leader_election.cc:304] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf9ed7972b524fa4a30b9c64e8b123c4, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:29.378026 25462 raft_consensus.cc:2804] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:29.378444 25462 raft_consensus.cc:697] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Leader. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:29.379292 25462 consensus_queue.cc:237] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:29.386240 25160 catalog_manager.cc:5697] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1017bddda0847b2867904967a83ef54 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "e1017bddda0847b2867904967a83ef54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.447504 25160 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43688:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:29.449501 25160 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:29.473964 25251 tablet_service.cc:1511] Processing CreateTablet for tablet bec52a2cebcc4cedb4bea27d03f4ce39 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.475432 25251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec52a2cebcc4cedb4bea27d03f4ce39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.475188 25327 tablet_service.cc:1511] Processing CreateTablet for tablet bec52a2cebcc4cedb4bea27d03f4ce39 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.476603 25327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec52a2cebcc4cedb4bea27d03f4ce39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.476320 25400 tablet_service.cc:1511] Processing CreateTablet for tablet bec52a2cebcc4cedb4bea27d03f4ce39 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:29.477689 25400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec52a2cebcc4cedb4bea27d03f4ce39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.478981 25252 tablet_service.cc:1511] Processing CreateTablet for tablet 55bb799b8e144c2cb5e8350098033cfd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.479372 25326 tablet_service.cc:1511] Processing CreateTablet for tablet 55bb799b8e144c2cb5e8350098033cfd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.480399 25252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bb799b8e144c2cb5e8350098033cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.480641 25326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bb799b8e144c2cb5e8350098033cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.494892 25452 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Bootstrap starting.
I20260709 13:07:29.500500 25455 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: Bootstrap starting.
I20260709 13:07:29.502810 25452 tablet_bootstrap.cc:654] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.502942 25400 tablet_service.cc:1511] Processing CreateTablet for tablet 55bb799b8e144c2cb5e8350098033cfd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a327e1acefc044b7adbc2404774c7279]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:29.504429 25400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bb799b8e144c2cb5e8350098033cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:29.507669 25455 tablet_bootstrap.cc:654] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.515933 25452 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: No bootstrap required, opened a new log
I20260709 13:07:29.516085 25451 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: Bootstrap starting.
I20260709 13:07:29.516673 25452 ts_tablet_manager.cc:1403] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:07:29.521260 25452 raft_consensus.cc:359] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.521786 25455 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: No bootstrap required, opened a new log
I20260709 13:07:29.522056 25452 raft_consensus.cc:385] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.522394 25452 raft_consensus.cc:740] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.523418 25452 consensus_queue.cc:260] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.522394 25455 ts_tablet_manager.cc:1403] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.012s
I20260709 13:07:29.525802 25452 ts_tablet_manager.cc:1434] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:29.527602 25451 tablet_bootstrap.cc:654] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.528664 25455 raft_consensus.cc:359] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.529518 25455 raft_consensus.cc:385] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.529860 25455 raft_consensus.cc:740] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ed7972b524fa4a30b9c64e8b123c4, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.530804 25455 consensus_queue.cc:260] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.535352 25455 ts_tablet_manager.cc:1434] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:29.536494 25455 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: Bootstrap starting.
I20260709 13:07:29.537871 25451 tablet_bootstrap.cc:492] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: No bootstrap required, opened a new log
I20260709 13:07:29.538447 25451 ts_tablet_manager.cc:1403] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 13:07:29.541661 25451 raft_consensus.cc:359] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.542482 25451 raft_consensus.cc:385] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.543231 25452 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: Bootstrap starting.
I20260709 13:07:29.543705 25455 tablet_bootstrap.cc:654] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.544775 25451 raft_consensus.cc:740] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0860048032459185b00d922c323bc5, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.545746 25451 consensus_queue.cc:260] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.548311 25451 ts_tablet_manager.cc:1434] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:29.549580 25451 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: Bootstrap starting.
I20260709 13:07:29.550290 25452 tablet_bootstrap.cc:654] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.556918 25451 tablet_bootstrap.cc:654] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:29.561331 25452 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: No bootstrap required, opened a new log
I20260709 13:07:29.561997 25452 ts_tablet_manager.cc:1403] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:07:29.564536 25455 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: No bootstrap required, opened a new log
I20260709 13:07:29.565285 25455 ts_tablet_manager.cc:1403] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.008s
I20260709 13:07:29.565354 25452 raft_consensus.cc:359] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.566103 25452 raft_consensus.cc:385] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.566354 25452 raft_consensus.cc:740] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.567261 25451 tablet_bootstrap.cc:492] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: No bootstrap required, opened a new log
I20260709 13:07:29.567059 25452 consensus_queue.cc:260] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.567816 25451 ts_tablet_manager.cc:1403] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:07:29.568572 25455 raft_consensus.cc:359] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.569447 25455 raft_consensus.cc:385] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.569526 25452 ts_tablet_manager.cc:1434] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:29.569890 25455 raft_consensus.cc:740] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ed7972b524fa4a30b9c64e8b123c4, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.571331 25451 raft_consensus.cc:359] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.571444 25455 consensus_queue.cc:260] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.572614 25451 raft_consensus.cc:385] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:29.573036 25451 raft_consensus.cc:740] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0860048032459185b00d922c323bc5, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.573873 25451 consensus_queue.cc:260] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.576416 25451 ts_tablet_manager.cc:1434] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:29.578370 25455 ts_tablet_manager.cc:1434] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 13:07:29.582826 25208 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:29.687461 25337 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.694803 25462 consensus_queue.cc:1048] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:29.705125 25462 consensus_queue.cc:1048] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:29.769564 25462 raft_consensus.cc:493] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.770188 25462 raft_consensus.cc:515] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.772845 25462 leader_election.cc:290] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.773190 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5" is_pre_election: true
I20260709 13:07:29.773588 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" is_pre_election: true
I20260709 13:07:29.774029 25262 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 0.
I20260709 13:07:29.774330 25337 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 0.
I20260709 13:07:29.775264 25374 leader_election.cc:304] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:29.776100 25462 raft_consensus.cc:2804] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:29.776480 25462 raft_consensus.cc:493] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.776821 25462 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.780565 25457 raft_consensus.cc:493] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.781217 25457 raft_consensus.cc:515] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.782778 25462 raft_consensus.cc:515] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.784024 25457 leader_election.cc:290] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311), e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853)
I20260709 13:07:29.785507 25462 leader_election.cc:290] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Requested vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:29.787881 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0860048032459185b00d922c323bc5"
I20260709 13:07:29.788807 25262 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.794813 25462 consensus_queue.cc:1048] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:29.797240 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:29.798110 25337 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.804337 25335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55bb799b8e144c2cb5e8350098033cfd" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" is_pre_election: true
I20260709 13:07:29.805190 25335 raft_consensus.cc:2468] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 0.
I20260709 13:07:29.806561 25226 leader_election.cc:304] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, cf9ed7972b524fa4a30b9c64e8b123c4; no voters: 
I20260709 13:07:29.806772 25411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55bb799b8e144c2cb5e8350098033cfd" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1017bddda0847b2867904967a83ef54" is_pre_election: true
I20260709 13:07:29.807750 25411 raft_consensus.cc:2468] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 0.
I20260709 13:07:29.808807 25457 raft_consensus.cc:2804] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:29.809350 25457 raft_consensus.cc:493] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.809756 25457 raft_consensus.cc:3060] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.811671 25337 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 1.
I20260709 13:07:29.806764 25262 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 1.
I20260709 13:07:29.813040 25375 leader_election.cc:304] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf9ed7972b524fa4a30b9c64e8b123c4, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:29.816555 25489 raft_consensus.cc:2804] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:29.817147 25489 raft_consensus.cc:697] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Leader. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:29.818084 25489 consensus_queue.cc:237] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.819024 25463 consensus_queue.cc:1048] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:29.819558 25457 raft_consensus.cc:515] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.821779 25457 leader_election.cc:290] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 1 election: Requested vote from peers cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311), e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853)
I20260709 13:07:29.822849 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55bb799b8e144c2cb5e8350098033cfd" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:29.823418 25337 raft_consensus.cc:3060] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.829664 25337 raft_consensus.cc:2468] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 1.
I20260709 13:07:29.831069 25226 leader_election.cc:304] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, cf9ed7972b524fa4a30b9c64e8b123c4; no voters: 
I20260709 13:07:29.832854 25457 raft_consensus.cc:2804] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:29.833348 25457 raft_consensus.cc:697] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 1 LEADER]: Becoming Leader. State: Replica: 7f0860048032459185b00d922c323bc5, State: Running, Role: LEADER
I20260709 13:07:29.834429 25457 consensus_queue.cc:237] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:29.836143 25411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55bb799b8e144c2cb5e8350098033cfd" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1017bddda0847b2867904967a83ef54"
I20260709 13:07:29.837210 25411 raft_consensus.cc:3060] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:29.848719 25159 catalog_manager.cc:5697] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1017bddda0847b2867904967a83ef54 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "e1017bddda0847b2867904967a83ef54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.850589 25411 raft_consensus.cc:2468] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 1.
I20260709 13:07:29.856981 25158 catalog_manager.cc:5697] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f0860048032459185b00d922c323bc5 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "7f0860048032459185b00d922c323bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.919625 25411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc"
dest_uuid: "e1017bddda0847b2867904967a83ef54"
mode: GRACEFUL
new_leader_uuid: "7f0860048032459185b00d922c323bc5"
 from {username='slave'} at 127.0.0.1:55948
I20260709 13:07:29.920208 25411 raft_consensus.cc:606] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Received request to transfer leadership to TS 7f0860048032459185b00d922c323bc5
I20260709 13:07:29.925984 25411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a33408c2a79749429762468f4f930b0e"
dest_uuid: "e1017bddda0847b2867904967a83ef54"
mode: GRACEFUL
new_leader_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
 from {username='slave'} at 127.0.0.1:55948
I20260709 13:07:29.926483 25411 raft_consensus.cc:606] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Received request to transfer leadership to TS cf9ed7972b524fa4a30b9c64e8b123c4
I20260709 13:07:29.932112 25411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39"
dest_uuid: "e1017bddda0847b2867904967a83ef54"
mode: GRACEFUL
new_leader_uuid: "7f0860048032459185b00d922c323bc5"
 from {username='slave'} at 127.0.0.1:55948
I20260709 13:07:29.932654 25411 raft_consensus.cc:606] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Received request to transfer leadership to TS 7f0860048032459185b00d922c323bc5
I20260709 13:07:30.209245 25462 raft_consensus.cc:993] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: : Instructing follower 7f0860048032459185b00d922c323bc5 to start an election
I20260709 13:07:30.209744 25462 raft_consensus.cc:1081] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Signalling peer 7f0860048032459185b00d922c323bc5 to start an election
I20260709 13:07:30.211342 25262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc"
dest_uuid: "7f0860048032459185b00d922c323bc5"
 from {username='slave'} at 127.0.0.1:56444
I20260709 13:07:30.211838 25262 raft_consensus.cc:493] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.212155 25262 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.217935 25262 raft_consensus.cc:515] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:30.220417 25262 leader_election.cc:290] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 2 election: Requested vote from peers e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:30.220598 25411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1017bddda0847b2867904967a83ef54"
I20260709 13:07:30.220813 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a88bbda569547a987b7da77d01b49cc" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:30.221271 25411 raft_consensus.cc:3055] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.221462 25337 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.221596 25411 raft_consensus.cc:740] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:30.222522 25411 consensus_queue.cc:260] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:30.223477 25411 raft_consensus.cc:3060] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.227640 25337 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 2.
I20260709 13:07:30.228747 25226 leader_election.cc:304] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, cf9ed7972b524fa4a30b9c64e8b123c4; no voters: 
I20260709 13:07:30.229043 25411 raft_consensus.cc:2468] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 2.
I20260709 13:07:30.229641 25457 raft_consensus.cc:2804] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.230100 25457 raft_consensus.cc:697] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Becoming Leader. State: Replica: 7f0860048032459185b00d922c323bc5, State: Running, Role: LEADER
I20260709 13:07:30.230937 25457 consensus_queue.cc:237] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } }
I20260709 13:07:30.237681 25158 catalog_manager.cc:5697] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 reported cstate change: term changed from 1 to 2, leader changed from e1017bddda0847b2867904967a83ef54 (127.13.163.195) to 7f0860048032459185b00d922c323bc5 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "7f0860048032459185b00d922c323bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.256283 25457 consensus_queue.cc:1048] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:30.257360 25462 consensus_queue.cc:1048] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:30.272557 25491 consensus_queue.cc:1048] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:30.273756 25498 consensus_queue.cc:1048] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:30.282931 25456 raft_consensus.cc:993] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: : Instructing follower 7f0860048032459185b00d922c323bc5 to start an election
I20260709 13:07:30.283529 25456 raft_consensus.cc:1081] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Signalling peer 7f0860048032459185b00d922c323bc5 to start an election
I20260709 13:07:30.284003 25489 raft_consensus.cc:993] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: : Instructing follower cf9ed7972b524fa4a30b9c64e8b123c4 to start an election
I20260709 13:07:30.284823 25489 raft_consensus.cc:1081] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Signalling peer cf9ed7972b524fa4a30b9c64e8b123c4 to start an election
I20260709 13:07:30.285460 25262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39"
dest_uuid: "7f0860048032459185b00d922c323bc5"
 from {username='slave'} at 127.0.0.1:56444
I20260709 13:07:30.286114 25262 raft_consensus.cc:493] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.286499 25262 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.286737 25337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a33408c2a79749429762468f4f930b0e"
dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
 from {username='slave'} at 127.0.0.1:44702
I20260709 13:07:30.287351 25337 raft_consensus.cc:493] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.287719 25337 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.295200 25262 raft_consensus.cc:515] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.297313 25337 raft_consensus.cc:515] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.299590 25262 leader_election.cc:290] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 2 election: Requested vote from peers cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311), e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853)
I20260709 13:07:30.301170 25410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1017bddda0847b2867904967a83ef54"
I20260709 13:07:30.301908 25410 raft_consensus.cc:3055] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.302271 25410 raft_consensus.cc:740] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:30.303092 25410 consensus_queue.cc:260] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.304543 25410 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.305296 25336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "7f0860048032459185b00d922c323bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:30.305938 25336 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.312279 25410 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 2.
I20260709 13:07:30.313674 25227 leader_election.cc:304] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:30.314651 25498 raft_consensus.cc:2804] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.314869 25336 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f0860048032459185b00d922c323bc5 in term 2.
I20260709 13:07:30.315167 25498 raft_consensus.cc:697] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Becoming Leader. State: Replica: 7f0860048032459185b00d922c323bc5, State: Running, Role: LEADER
I20260709 13:07:30.316397 25498 consensus_queue.cc:237] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.317471 25337 leader_election.cc:290] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [CANDIDATE]: Term 2 election: Requested vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), e1017bddda0847b2867904967a83ef54 (127.13.163.195:36853)
I20260709 13:07:30.325871 25159 catalog_manager.cc:5697] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 reported cstate change: term changed from 1 to 2, leader changed from e1017bddda0847b2867904967a83ef54 (127.13.163.195) to 7f0860048032459185b00d922c323bc5 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "7f0860048032459185b00d922c323bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.329774 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f0860048032459185b00d922c323bc5"
I20260709 13:07:30.329751 25410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a33408c2a79749429762468f4f930b0e" candidate_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1017bddda0847b2867904967a83ef54"
I20260709 13:07:30.330632 25262 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.330690 25410 raft_consensus.cc:3055] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.331153 25410 raft_consensus.cc:740] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:30.331996 25410 consensus_queue.cc:260] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.333307 25410 raft_consensus.cc:3060] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.338153 25262 raft_consensus.cc:2468] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf9ed7972b524fa4a30b9c64e8b123c4 in term 2.
I20260709 13:07:30.339533 25300 leader_election.cc:304] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f0860048032459185b00d922c323bc5, cf9ed7972b524fa4a30b9c64e8b123c4; no voters: 
I20260709 13:07:30.340040 25410 raft_consensus.cc:2468] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf9ed7972b524fa4a30b9c64e8b123c4 in term 2.
I20260709 13:07:30.340399 25464 raft_consensus.cc:2804] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.341552 25464 raft_consensus.cc:697] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 LEADER]: Becoming Leader. State: Replica: cf9ed7972b524fa4a30b9c64e8b123c4, State: Running, Role: LEADER
I20260709 13:07:30.342312 25464 consensus_queue.cc:237] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.348670 25159 catalog_manager.cc:5697] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 reported cstate change: term changed from 1 to 2, leader changed from e1017bddda0847b2867904967a83ef54 (127.13.163.195) to cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.375041 13967 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:07:30.376624 13967 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:07:30.377053 13967 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:07:30.388757 25262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39"
dest_uuid: "7f0860048032459185b00d922c323bc5"
mode: GRACEFUL
new_leader_uuid: "e1017bddda0847b2867904967a83ef54"
 from {username='slave'} at 127.0.0.1:56470
I20260709 13:07:30.389362 25262 raft_consensus.cc:606] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Received request to transfer leadership to TS e1017bddda0847b2867904967a83ef54
I20260709 13:07:30.390558 13967 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:07:30.618814 25337 raft_consensus.cc:1275] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f0860048032459185b00d922c323bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.620136 25498 consensus_queue.cc:1048] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.628062 25410 raft_consensus.cc:1275] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Refusing update from remote peer 7f0860048032459185b00d922c323bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.630043 25457 consensus_queue.cc:1048] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.722651 25337 raft_consensus.cc:1275] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f0860048032459185b00d922c323bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.724270 25457 consensus_queue.cc:1048] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.727820 25410 raft_consensus.cc:1275] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Refusing update from remote peer 7f0860048032459185b00d922c323bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.729239 25498 consensus_queue.cc:1048] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.738796 25492 raft_consensus.cc:993] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: : Instructing follower e1017bddda0847b2867904967a83ef54 to start an election
I20260709 13:07:30.739337 25492 raft_consensus.cc:1081] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Signalling peer e1017bddda0847b2867904967a83ef54 to start an election
I20260709 13:07:30.744134 25410 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39"
dest_uuid: "e1017bddda0847b2867904967a83ef54"
 from {username='slave'} at 127.0.0.1:55946
I20260709 13:07:30.744901 25410 raft_consensus.cc:493] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.746981 25410 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.754170 25410 raft_consensus.cc:515] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.755997 25410 leader_election.cc:290] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 3 election: Requested vote from peers 7f0860048032459185b00d922c323bc5 (127.13.163.193:35739), cf9ed7972b524fa4a30b9c64e8b123c4 (127.13.163.194:34311)
I20260709 13:07:30.756810 25262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f0860048032459185b00d922c323bc5"
I20260709 13:07:30.757153 25337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec52a2cebcc4cedb4bea27d03f4ce39" candidate_uuid: "e1017bddda0847b2867904967a83ef54" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4"
I20260709 13:07:30.757589 25262 raft_consensus.cc:3055] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:30.757793 25337 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.757983 25262 raft_consensus.cc:740] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f0860048032459185b00d922c323bc5, State: Running, Role: LEADER
I20260709 13:07:30.758953 25262 consensus_queue.cc:260] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.760057 25262 raft_consensus.cc:3060] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.763321 25337 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 3.
I20260709 13:07:30.764747 25375 leader_election.cc:304] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf9ed7972b524fa4a30b9c64e8b123c4, e1017bddda0847b2867904967a83ef54; no voters: 
I20260709 13:07:30.765676 25489 raft_consensus.cc:2804] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:30.766000 25262 raft_consensus.cc:2468] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1017bddda0847b2867904967a83ef54 in term 3.
I20260709 13:07:30.766122 25489 raft_consensus.cc:697] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 LEADER]: Becoming Leader. State: Replica: e1017bddda0847b2867904967a83ef54, State: Running, Role: LEADER
I20260709 13:07:30.767133 25489 consensus_queue.cc:237] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } }
I20260709 13:07:30.773386 25160 catalog_manager.cc:5697] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 reported cstate change: term changed from 2 to 3, leader changed from 7f0860048032459185b00d922c323bc5 (127.13.163.193) to e1017bddda0847b2867904967a83ef54 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "e1017bddda0847b2867904967a83ef54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9ed7972b524fa4a30b9c64e8b123c4" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.812108 25410 raft_consensus.cc:1275] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Refusing update from remote peer cf9ed7972b524fa4a30b9c64e8b123c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.813359 25464 consensus_queue.cc:1048] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1017bddda0847b2867904967a83ef54" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.822824 25262 raft_consensus.cc:1275] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Refusing update from remote peer cf9ed7972b524fa4a30b9c64e8b123c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.824937 25507 consensus_queue.cc:1048] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0860048032459185b00d922c323bc5" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:31.016094 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
I20260709 13:07:31.036667 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:31.037345 13967 tablet_replica.cc:333] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5: stopping tablet replica
I20260709 13:07:31.037914 13967 raft_consensus.cc:2243] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.038738 13967 raft_consensus.cc:2272] T 55bb799b8e144c2cb5e8350098033cfd P 7f0860048032459185b00d922c323bc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.041118 13967 tablet_replica.cc:333] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5: stopping tablet replica
I20260709 13:07:31.041644 13967 raft_consensus.cc:2243] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.042104 13967 raft_consensus.cc:2272] T bec52a2cebcc4cedb4bea27d03f4ce39 P 7f0860048032459185b00d922c323bc5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.044932 13967 tablet_replica.cc:333] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5: stopping tablet replica
I20260709 13:07:31.045450 13967 raft_consensus.cc:2243] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.046262 13967 raft_consensus.cc:2272] T 7a88bbda569547a987b7da77d01b49cc P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.048496 13967 tablet_replica.cc:333] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5: stopping tablet replica
I20260709 13:07:31.049052 13967 raft_consensus.cc:2243] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.049458 13967 raft_consensus.cc:2272] T a33408c2a79749429762468f4f930b0e P 7f0860048032459185b00d922c323bc5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.072257 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
I20260709 13:07:31.084707 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
I20260709 13:07:31.107218 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:31.107959 13967 tablet_replica.cc:333] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4: stopping tablet replica
I20260709 13:07:31.108520 13967 raft_consensus.cc:2243] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.109028 13967 raft_consensus.cc:2272] T 55bb799b8e144c2cb5e8350098033cfd P cf9ed7972b524fa4a30b9c64e8b123c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.111531 13967 tablet_replica.cc:333] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4: stopping tablet replica
I20260709 13:07:31.112062 13967 raft_consensus.cc:2243] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.112474 13967 raft_consensus.cc:2272] T bec52a2cebcc4cedb4bea27d03f4ce39 P cf9ed7972b524fa4a30b9c64e8b123c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.114804 13967 tablet_replica.cc:333] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4: stopping tablet replica
I20260709 13:07:31.115334 13967 raft_consensus.cc:2243] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.116150 13967 raft_consensus.cc:2272] T a33408c2a79749429762468f4f930b0e P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.118485 13967 tablet_replica.cc:333] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4: stopping tablet replica
I20260709 13:07:31.118963 13967 raft_consensus.cc:2243] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.119403 13967 raft_consensus.cc:2272] T 7a88bbda569547a987b7da77d01b49cc P cf9ed7972b524fa4a30b9c64e8b123c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.145674 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
I20260709 13:07:31.157615 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
I20260709 13:07:31.174997 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:31.175727 13967 tablet_replica.cc:333] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54: stopping tablet replica
I20260709 13:07:31.176296 13967 raft_consensus.cc:2243] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.176777 13967 raft_consensus.cc:2272] T 55bb799b8e144c2cb5e8350098033cfd P e1017bddda0847b2867904967a83ef54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.179373 13967 tablet_replica.cc:333] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: stopping tablet replica
I20260709 13:07:31.179919 13967 raft_consensus.cc:2243] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.180562 13967 pending_rounds.cc:70] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Trying to abort 1 pending ops.
I20260709 13:07:31.180778 13967 pending_rounds.cc:77] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635638349623296 op_type: NO_OP noop_request { }
I20260709 13:07:31.181142 13967 raft_consensus.cc:2889] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:31.181427 13967 raft_consensus.cc:2272] T bec52a2cebcc4cedb4bea27d03f4ce39 P e1017bddda0847b2867904967a83ef54 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.183580 13967 tablet_replica.cc:333] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54: stopping tablet replica
I20260709 13:07:31.184080 13967 raft_consensus.cc:2243] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.184480 13967 raft_consensus.cc:2272] T 7a88bbda569547a987b7da77d01b49cc P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.186703 13967 tablet_replica.cc:333] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54: stopping tablet replica
I20260709 13:07:31.187178 13967 raft_consensus.cc:2243] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.187578 13967 raft_consensus.cc:2272] T a33408c2a79749429762468f4f930b0e P e1017bddda0847b2867904967a83ef54 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.210386 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:07:31.222419 13967 master.cc:562] Master@127.13.163.254:39159 shutting down...
I20260709 13:07:31.238168 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.238757 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.239143 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 942cc6adfc4a41e0bdc144b4001e1eb6: stopping tablet replica
I20260709 13:07:31.257308 13967 master.cc:584] Master@127.13.163.254:39159 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2850 ms)
[----------] 6 tests from LeaderRebalancerTest (358682 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:07:31.286911 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:37557
I20260709 13:07:31.287914 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:31.293182 25509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.293205 25510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.294617 25512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:31.295887 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:31.296818 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:31.297077 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:31.297250 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602451297232 us; error 0 us; skew 500 ppm
I20260709 13:07:31.297791 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:31.300168 13967 webserver.cc:533] Webserver started at http://127.13.163.254:34745/ using document root <none> and password file <none>
I20260709 13:07:31.300661 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:31.300886 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:31.301162 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:31.302206 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "a5e911fd20f24bcdbe43b8cb02ba0948"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.306761 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:31.310092 25517 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.310883 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:31.311164 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "a5e911fd20f24bcdbe43b8cb02ba0948"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.311455 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:31.340600 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:31.341677 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:31.376722 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:37557
I20260709 13:07:31.376914 25568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:37557 every 8 connection(s)
I20260709 13:07:31.380520 25569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.391857 25569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: Bootstrap starting.
I20260709 13:07:31.396507 25569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:31.400574 25569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: No bootstrap required, opened a new log
I20260709 13:07:31.402680 25569 raft_consensus.cc:359] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER }
I20260709 13:07:31.403090 25569 raft_consensus.cc:385] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:31.403530 25569 raft_consensus.cc:740] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e911fd20f24bcdbe43b8cb02ba0948, State: Initialized, Role: FOLLOWER
I20260709 13:07:31.404279 25569 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER }
I20260709 13:07:31.404760 25569 raft_consensus.cc:399] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:31.405083 25569 raft_consensus.cc:493] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:31.405375 25569 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:31.410277 25569 raft_consensus.cc:515] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER }
I20260709 13:07:31.410892 25569 leader_election.cc:304] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5e911fd20f24bcdbe43b8cb02ba0948; no voters: 
I20260709 13:07:31.412048 25569 leader_election.cc:290] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:31.412390 25572 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:31.413755 25572 raft_consensus.cc:697] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 1 LEADER]: Becoming Leader. State: Replica: a5e911fd20f24bcdbe43b8cb02ba0948, State: Running, Role: LEADER
I20260709 13:07:31.414395 25572 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER }
I20260709 13:07:31.415159 25569 sys_catalog.cc:565] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:31.417099 25574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5e911fd20f24bcdbe43b8cb02ba0948. Latest consensus state: current_term: 1 leader_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER } }
I20260709 13:07:31.417093 25573 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5e911fd20f24bcdbe43b8cb02ba0948" member_type: VOTER } }
I20260709 13:07:31.417833 25574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:31.417925 25573 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:31.421747 25576 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:31.426344 25576 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:31.430810 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:31.436560 25576 catalog_manager.cc:1383] Generated new cluster ID: 6f3ed7d417254f3e9c245b6bf40757ec
I20260709 13:07:31.436884 25576 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:31.506158 25576 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:31.507566 25576 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:31.521046 25576 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: Generated new TSK 0
I20260709 13:07:31.521728 25576 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:31.561817 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:31.567764 25590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.568794 25591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.570487 25593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:31.570744 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:31.571630 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:31.571823 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:31.571982 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602451571959 us; error 0 us; skew 500 ppm
I20260709 13:07:31.572575 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:31.574965 13967 webserver.cc:533] Webserver started at http://127.13.163.193:35459/ using document root <none> and password file <none>
I20260709 13:07:31.575438 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:31.575610 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:31.575829 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:31.576928 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "466dd86270c5461ca8e12b502390e16a"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.581378 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:07:31.584798 25598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.585647 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:31.585969 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "466dd86270c5461ca8e12b502390e16a"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.586247 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:31.599284 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:31.600495 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:31.602016 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:31.604290 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:31.604486 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.604787 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:31.605026 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:31.645886 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:43187
I20260709 13:07:31.646024 25660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:43187 every 8 connection(s)
I20260709 13:07:31.650760 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:31.658052 25665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.658948 25666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:31.664564 13967 server_base.cc:1061] running on GCE node
W20260709 13:07:31.667083 25668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:31.667433 25661 heartbeater.cc:344] Connected to a master server at 127.13.163.254:37557
I20260709 13:07:31.668042 25661 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:31.669219 25661 heartbeater.cc:507] Master 127.13.163.254:37557 requested a full tablet report, sending...
I20260709 13:07:31.671870 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:31.671967 25534 ts_manager.cc:194] Registered new tserver with Master: 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187)
W20260709 13:07:31.672242 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:31.672559 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602451672543 us; error 0 us; skew 500 ppm
I20260709 13:07:31.673223 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:31.674851 25534 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52832
I20260709 13:07:31.676993 13967 webserver.cc:533] Webserver started at http://127.13.163.194:39661/ using document root <none> and password file <none>
I20260709 13:07:31.677565 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:31.677778 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:31.678042 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:31.679126 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.683822 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:07:31.687517 25673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.688364 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:31.688688 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.689054 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:31.716287 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:31.717473 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:31.718899 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:31.721221 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:31.721448 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.721688 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:31.721870 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.762039 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:33649
I20260709 13:07:31.762176 25735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:33649 every 8 connection(s)
I20260709 13:07:31.766806 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:31.774217 25739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.776106 25740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:31.778723 25742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:31.778774 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:31.780133 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:31.780470 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:31.780548 25736 heartbeater.cc:344] Connected to a master server at 127.13.163.254:37557
I20260709 13:07:31.780689 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602451780671 us; error 0 us; skew 500 ppm
I20260709 13:07:31.781214 25736 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:31.781589 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:31.782009 25736 heartbeater.cc:507] Master 127.13.163.254:37557 requested a full tablet report, sending...
I20260709 13:07:31.784303 25534 ts_manager.cc:194] Registered new tserver with Master: 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:31.784361 13967 webserver.cc:533] Webserver started at http://127.13.163.195:40451/ using document root <none> and password file <none>
I20260709 13:07:31.785089 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:31.785292 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:31.785584 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:31.785955 25534 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52848
I20260709 13:07:31.786707 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.791706 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:07:31.795321 25747 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.796119 13967 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:31.796455 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
format_stamp: "Formatted at 2026-07-09 13:07:31 on dist-test-slave-w3b1"
I20260709 13:07:31.796700 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:31.808120 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:31.809214 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:31.810493 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:31.812644 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:31.812903 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.813185 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:31.813393 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:31.855737 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:33817
I20260709 13:07:31.855858 25809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:33817 every 8 connection(s)
I20260709 13:07:31.869174 25810 heartbeater.cc:344] Connected to a master server at 127.13.163.254:37557
I20260709 13:07:31.869658 25810 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:31.870405 25810 heartbeater.cc:507] Master 127.13.163.254:37557 requested a full tablet report, sending...
I20260709 13:07:31.872608 25534 ts_manager.cc:194] Registered new tserver with Master: fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:31.873409 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014503838s
I20260709 13:07:31.874322 25534 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52850
I20260709 13:07:31.900189 25534 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52854:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:31.903015 25534 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:31.981462 25701 tablet_service.cc:1511] Processing CreateTablet for tablet ddbb0f118713403e92a23cf7f1d71eda (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:31.982910 25701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbb0f118713403e92a23cf7f1d71eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.982597 25700 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe634984c1149f78138ccff0f419bcf (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:31.984192 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe634984c1149f78138ccff0f419bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.985707 25699 tablet_service.cc:1511] Processing CreateTablet for tablet daae2603a8ad48b4ad155c7971a07739 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:31.986789 25696 tablet_service.cc:1511] Processing CreateTablet for tablet fca2c18175044c57b773b2f27526fe2f (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:31.988184 25626 tablet_service.cc:1511] Processing CreateTablet for tablet ddbb0f118713403e92a23cf7f1d71eda (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:31.989745 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbb0f118713403e92a23cf7f1d71eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.989444 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdbf339b8f941baaffa2d12334932f0 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:31.991544 25698 tablet_service.cc:1511] Processing CreateTablet for tablet 902c2b59aab64796be18c4dc3adaca06 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:31.993181 25622 tablet_service.cc:1511] Processing CreateTablet for tablet 175d87f9e1d04e73bf1464b3070e7370 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:31.994055 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe634984c1149f78138ccff0f419bcf (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:31.994565 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175d87f9e1d04e73bf1464b3070e7370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.995488 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe634984c1149f78138ccff0f419bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:31.997177 25694 tablet_service.cc:1511] Processing CreateTablet for tablet b3fb62cb171046f1a45df62a2c4efcff (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:31.999123 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 175d87f9e1d04e73bf1464b3070e7370 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:32.001592 25624 tablet_service.cc:1511] Processing CreateTablet for tablet daae2603a8ad48b4ad155c7971a07739 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:32.002338 25693 tablet_service.cc:1511] Processing CreateTablet for tablet 5f1414dc93b84fa49258bd56a9334f65 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:32.003132 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daae2603a8ad48b4ad155c7971a07739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.004769 25623 tablet_service.cc:1511] Processing CreateTablet for tablet 902c2b59aab64796be18c4dc3adaca06 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:32.006642 25623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c2b59aab64796be18c4dc3adaca06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.010519 25621 tablet_service.cc:1511] Processing CreateTablet for tablet fca2c18175044c57b773b2f27526fe2f (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:32.011103 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fb62cb171046f1a45df62a2c4efcff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.017521 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daae2603a8ad48b4ad155c7971a07739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.017196 25618 tablet_service.cc:1511] Processing CreateTablet for tablet 5f1414dc93b84fa49258bd56a9334f65 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:32.020035 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2c18175044c57b773b2f27526fe2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.021152 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdbf339b8f941baaffa2d12334932f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.012238 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2c18175044c57b773b2f27526fe2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.025198 25618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f1414dc93b84fa49258bd56a9334f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.013661 25620 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdbf339b8f941baaffa2d12334932f0 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:32.015425 25619 tablet_service.cc:1511] Processing CreateTablet for tablet b3fb62cb171046f1a45df62a2c4efcff (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:32.028043 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdbf339b8f941baaffa2d12334932f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.033366 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c2b59aab64796be18c4dc3adaca06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.033502 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fb62cb171046f1a45df62a2c4efcff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.037356 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175d87f9e1d04e73bf1464b3070e7370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.042016 25693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f1414dc93b84fa49258bd56a9334f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.071334 25825 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.074582 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 902c2b59aab64796be18c4dc3adaca06 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:32.076085 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c2b59aab64796be18c4dc3adaca06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.075915 25769 tablet_service.cc:1511] Processing CreateTablet for tablet 9bdbf339b8f941baaffa2d12334932f0 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:32.077366 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bdbf339b8f941baaffa2d12334932f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.077957 25771 tablet_service.cc:1511] Processing CreateTablet for tablet 175d87f9e1d04e73bf1464b3070e7370 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:32.079360 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175d87f9e1d04e73bf1464b3070e7370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.081928 25770 tablet_service.cc:1511] Processing CreateTablet for tablet fca2c18175044c57b773b2f27526fe2f (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:32.083348 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2c18175044c57b773b2f27526fe2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.084684 25775 tablet_service.cc:1511] Processing CreateTablet for tablet ddbb0f118713403e92a23cf7f1d71eda (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:32.081928 25773 tablet_service.cc:1511] Processing CreateTablet for tablet daae2603a8ad48b4ad155c7971a07739 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:32.082981 25774 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe634984c1149f78138ccff0f419bcf (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:32.085974 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbb0f118713403e92a23cf7f1d71eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.102620 25768 tablet_service.cc:1511] Processing CreateTablet for tablet b3fb62cb171046f1a45df62a2c4efcff (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:32.111223 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daae2603a8ad48b4ad155c7971a07739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.117219 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe634984c1149f78138ccff0f419bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.122741 25767 tablet_service.cc:1511] Processing CreateTablet for tablet 5f1414dc93b84fa49258bd56a9334f65 (DEFAULT_TABLE table=test-workload [id=aac7f7d84d6e463fa1d4590e8baaaaaf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:32.124614 25767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f1414dc93b84fa49258bd56a9334f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.129323 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fb62cb171046f1a45df62a2c4efcff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.147454 25826 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.154167 25825 tablet_bootstrap.cc:654] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.200268 25825 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.200826 25825 ts_tablet_manager.cc:1403] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.130s	user 0.013s	sys 0.005s
I20260709 13:07:32.203688 25825 raft_consensus.cc:359] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.204412 25825 raft_consensus.cc:385] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.204713 25825 raft_consensus.cc:740] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.205830 25826 tablet_bootstrap.cc:654] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.206346 25827 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.206066 25825 consensus_queue.cc:260] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.212280 25827 tablet_bootstrap.cc:654] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.223805 25736 heartbeater.cc:499] Master 127.13.163.254:37557 was elected leader, sending a full tablet report...
I20260709 13:07:32.231623 25826 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.232244 25826 ts_tablet_manager.cc:1403] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.014s
I20260709 13:07:32.233722 25825 ts_tablet_manager.cc:1434] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.033s	user 0.013s	sys 0.004s
I20260709 13:07:32.234606 25825 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.235147 25826 raft_consensus.cc:359] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.235981 25826 raft_consensus.cc:385] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.236304 25826 raft_consensus.cc:740] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.237217 25826 consensus_queue.cc:260] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.237797 25827 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.238479 25827 ts_tablet_manager.cc:1403] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.005s
I20260709 13:07:32.240993 25825 tablet_bootstrap.cc:654] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.241406 25827 raft_consensus.cc:359] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.242225 25827 raft_consensus.cc:385] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.242565 25827 raft_consensus.cc:740] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.243393 25827 consensus_queue.cc:260] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.246680 25826 ts_tablet_manager.cc:1434] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.006s
I20260709 13:07:32.247429 25661 heartbeater.cc:499] Master 127.13.163.254:37557 was elected leader, sending a full tablet report...
I20260709 13:07:32.252602 25826 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.263541 25810 heartbeater.cc:499] Master 127.13.163.254:37557 was elected leader, sending a full tablet report...
I20260709 13:07:32.263674 25827 ts_tablet_manager.cc:1434] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.017s
I20260709 13:07:32.265435 25827 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.268345 25826 tablet_bootstrap.cc:654] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.273639 25827 tablet_bootstrap.cc:654] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.276552 25825 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.277163 25825 ts_tablet_manager.cc:1403] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.025s
I20260709 13:07:32.280611 25825 raft_consensus.cc:359] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.281662 25825 raft_consensus.cc:385] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.282035 25825 raft_consensus.cc:740] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.283249 25825 consensus_queue.cc:260] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.286505 25825 ts_tablet_manager.cc:1434] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:32.287807 25825 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.291301 25827 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.291898 25827 ts_tablet_manager.cc:1403] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.016s
I20260709 13:07:32.294866 25827 raft_consensus.cc:359] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.295338 25825 tablet_bootstrap.cc:654] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.295671 25827 raft_consensus.cc:385] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.296015 25827 raft_consensus.cc:740] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.296972 25827 consensus_queue.cc:260] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.299964 25827 ts_tablet_manager.cc:1434] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:07:32.301308 25827 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.309616 25827 tablet_bootstrap.cc:654] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.322636 25827 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.323351 25827 ts_tablet_manager.cc:1403] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:07:32.327333 25827 raft_consensus.cc:359] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.328362 25827 raft_consensus.cc:385] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.328748 25827 raft_consensus.cc:740] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.329926 25827 consensus_queue.cc:260] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.332983 25827 ts_tablet_manager.cc:1434] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 13:07:32.334327 25827 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.342883 25827 tablet_bootstrap.cc:654] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.350373 25826 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.350875 25826 ts_tablet_manager.cc:1403] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.098s	user 0.023s	sys 0.054s
I20260709 13:07:32.353621 25826 raft_consensus.cc:359] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.354444 25826 raft_consensus.cc:385] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.354707 25826 raft_consensus.cc:740] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.354765 25827 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.355388 25827 ts_tablet_manager.cc:1403] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 13:07:32.355787 25825 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.355458 25826 consensus_queue.cc:260] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.356325 25825 ts_tablet_manager.cc:1403] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.069s	user 0.032s	sys 0.024s
I20260709 13:07:32.357723 25826 ts_tablet_manager.cc:1434] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:07:32.358734 25826 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.358435 25827 raft_consensus.cc:359] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.359207 25827 raft_consensus.cc:385] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.359589 25834 raft_consensus.cc:493] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.359674 25827 raft_consensus.cc:740] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.360234 25834 raft_consensus.cc:515] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.359156 25825 raft_consensus.cc:359] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.360826 25827 consensus_queue.cc:260] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.361557 25825 raft_consensus.cc:385] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.361884 25825 raft_consensus.cc:740] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.362964 25825 consensus_queue.cc:260] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.365465 25826 tablet_bootstrap.cc:654] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.366348 25825 ts_tablet_manager.cc:1434] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:07:32.367666 25825 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.371872 25827 ts_tablet_manager.cc:1434] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:07:32.374366 25825 tablet_bootstrap.cc:654] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.374881 25827 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.376899 25826 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.377529 25826 ts_tablet_manager.cc:1403] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.008s
I20260709 13:07:32.377400 25832 raft_consensus.cc:493] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.377980 25832 raft_consensus.cc:515] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.382668 25832 leader_election.cc:290] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.383849 25827 tablet_bootstrap.cc:654] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.388163 25825 tablet_bootstrap.cc:492] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.388901 25825 ts_tablet_manager.cc:1403] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 13:07:32.392234 25829 raft_consensus.cc:493] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.392923 25829 raft_consensus.cc:515] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.395596 25827 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.396181 25827 ts_tablet_manager.cc:1403] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.007s
I20260709 13:07:32.398248 25826 raft_consensus.cc:359] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.399036 25826 raft_consensus.cc:385] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.399261 25827 raft_consensus.cc:359] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.400269 25827 raft_consensus.cc:385] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.400626 25827 raft_consensus.cc:740] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.401748 25827 consensus_queue.cc:260] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.405078 25827 ts_tablet_manager.cc:1434] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:07:32.405603 25834 leader_election.cc:290] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.406409 25827 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.399327 25826 raft_consensus.cc:740] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.408946 25829 leader_election.cc:290] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817), 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187)
I20260709 13:07:32.410286 25826 consensus_queue.cc:260] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.415086 25827 tablet_bootstrap.cc:654] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.417583 25826 ts_tablet_manager.cc:1434] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.004s
I20260709 13:07:32.418969 25826 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
W20260709 13:07:32.430258 25582 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:32.431524 25826 tablet_bootstrap.cc:654] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.435477 25827 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.436059 25827 ts_tablet_manager.cc:1403] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.007s
I20260709 13:07:32.438203 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.439033 25636 raft_consensus.cc:2468] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.443163 25827 raft_consensus.cc:359] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.443980 25827 raft_consensus.cc:385] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.444283 25827 raft_consensus.cc:740] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.445084 25827 consensus_queue.cc:260] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.448050 25748 leader_election.cc:304] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.450858 25825 raft_consensus.cc:359] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.451813 25825 raft_consensus.cc:385] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.452163 25825 raft_consensus.cc:740] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.453102 25825 consensus_queue.cc:260] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.455561 25825 ts_tablet_manager.cc:1434] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.001s
I20260709 13:07:32.457605 25840 raft_consensus.cc:2804] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.458137 25840 raft_consensus.cc:493] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.458554 25840 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.460903 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.461747 25785 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:32.462515 25827 ts_tablet_manager.cc:1434] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:07:32.463181 25602 leader_election.cc:304] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.463721 25827 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.464104 25832 raft_consensus.cc:2804] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.464524 25832 raft_consensus.cc:493] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.464887 25832 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.469964 25827 tablet_bootstrap.cc:654] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.472335 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.473140 25711 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:32.474848 25840 raft_consensus.cc:515] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.472311 25832 raft_consensus.cc:515] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.479509 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:32.480059 25711 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.480382 25832 leader_election.cc:290] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.480592 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:32.482028 25636 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.486569 25711 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:32.487910 25601 leader_election.cc:304] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:32.488991 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:32.489725 25785 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.490682 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
W20260709 13:07:32.492365 25750 leader_election.cc:343] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.493266 25825 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.493352 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
W20260709 13:07:32.495534 25750 leader_election.cc:343] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.496078 25832 raft_consensus.cc:2804] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.498631 25827 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.499250 25827 ts_tablet_manager.cc:1403] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.014s
I20260709 13:07:32.500989 25829 raft_consensus.cc:493] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.501565 25829 raft_consensus.cc:515] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.503203 25826 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.502732 25827 raft_consensus.cc:359] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.503403 25825 tablet_bootstrap.cc:654] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.507957 25636 raft_consensus.cc:2468] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.503834 25826 ts_tablet_manager.cc:1403] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.007s
I20260709 13:07:32.504261 25829 leader_election.cc:290] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.512650 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.513559 25826 raft_consensus.cc:359] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.514458 25826 raft_consensus.cc:385] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.514803 25826 raft_consensus.cc:740] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
W20260709 13:07:32.515563 25674 leader_election.cc:343] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.513456 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.515734 25826 consensus_queue.cc:260] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
W20260709 13:07:32.518016 25674 leader_election.cc:343] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.481290 25840 leader_election.cc:290] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.503686 25827 raft_consensus.cc:385] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.519641 25827 raft_consensus.cc:740] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.520793 25827 consensus_queue.cc:260] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.500597 25785 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:32.526347 25748 leader_election.cc:304] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:32.527073 25827 ts_tablet_manager.cc:1434] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 13:07:32.527820 25840 raft_consensus.cc:2804] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.528729 25840 raft_consensus.cc:697] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:32.528823 25827 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.530066 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.529786 25840 consensus_queue.cc:237] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.530799 25784 raft_consensus.cc:2468] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:32.531821 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.534412 25677 leader_election.cc:304] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:32.535887 25829 raft_consensus.cc:2804] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.536329 25829 raft_consensus.cc:493] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.536751 25829 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.539693 25827 tablet_bootstrap.cc:654] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:32.541461 25677 leader_election.cc:343] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.542032 25677 leader_election.cc:304] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:32.550735 25826 ts_tablet_manager.cc:1434] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.040s	user 0.021s	sys 0.017s
I20260709 13:07:32.552695 25826 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.553319 25840 raft_consensus.cc:493] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.553969 25840 raft_consensus.cc:515] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.557055 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.557981 25636 raft_consensus.cc:2468] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.559859 25748 leader_election.cc:304] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.560727 25834 raft_consensus.cc:2804] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.561116 25834 raft_consensus.cc:493] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.561455 25834 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.551863 25530 catalog_manager.cc:5697] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 1 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.563607 25826 tablet_bootstrap.cc:654] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.567567 25829 raft_consensus.cc:515] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.569726 25858 raft_consensus.cc:2749] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:32.572131 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
W20260709 13:07:32.573798 25750 leader_election.cc:343] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.574712 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:32.575618 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:32.576231 25785 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:32.576309 25674 leader_election.cc:343] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.576998 25840 leader_election.cc:290] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.578368 25826 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.579213 25826 ts_tablet_manager.cc:1403] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.006s
I20260709 13:07:32.569998 25834 raft_consensus.cc:515] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.577502 25832 raft_consensus.cc:697] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:32.580377 25832 consensus_queue.cc:237] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.582226 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:32.582446 25834 leader_election.cc:290] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.582850 25636 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.582873 25826 raft_consensus.cc:359] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.583918 25826 raft_consensus.cc:385] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.584192 25785 raft_consensus.cc:2468] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:32.584275 25826 raft_consensus.cc:740] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.585673 25677 leader_election.cc:304] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:32.585448 25826 consensus_queue.cc:260] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.586658 25858 raft_consensus.cc:2804] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.588627 25826 ts_tablet_manager.cc:1434] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:32.588791 25827 tablet_bootstrap.cc:492] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.589464 25827 ts_tablet_manager.cc:1403] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.061s	user 0.035s	sys 0.018s
I20260709 13:07:32.589906 25826 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.589815 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
W20260709 13:07:32.591645 25750 leader_election.cc:343] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.591674 25825 tablet_bootstrap.cc:492] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.592974 25825 ts_tablet_manager.cc:1403] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.006s
I20260709 13:07:32.593937 25829 leader_election.cc:290] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.596360 25825 raft_consensus.cc:359] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.597136 25825 raft_consensus.cc:385] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.594017 25827 raft_consensus.cc:359] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.598759 25827 raft_consensus.cc:385] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.599043 25827 raft_consensus.cc:740] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.599555 25826 tablet_bootstrap.cc:654] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.600070 25636 raft_consensus.cc:2468] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.597452 25825 raft_consensus.cc:740] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.601675 25748 leader_election.cc:304] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:32.601792 25825 consensus_queue.cc:260] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.603191 25834 raft_consensus.cc:2804] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.605443 25834 raft_consensus.cc:697] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:32.606173 25840 raft_consensus.cc:493] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.606799 25840 raft_consensus.cc:515] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.606760 25834 consensus_queue.cc:237] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.609757 25840 leader_election.cc:290] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.610348 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.610857 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.604419 25825 ts_tablet_manager.cc:1434] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:07:32.615654 25825 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
W20260709 13:07:32.617060 25750 leader_election.cc:343] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.599748 25827 consensus_queue.cc:260] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.595722 25858 raft_consensus.cc:697] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:32.621404 25827 ts_tablet_manager.cc:1434] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.003s
W20260709 13:07:32.622231 25748 leader_election.cc:343] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.623170 25858 consensus_queue.cc:237] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.625203 25748 leader_election.cc:304] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:32.622689 25826 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.627068 25826 ts_tablet_manager.cc:1403] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.004s
I20260709 13:07:32.628216 25827 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:32.628801 25840 raft_consensus.cc:2749] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:32.626787 25533 catalog_manager.cc:5697] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.632885 25826 raft_consensus.cc:359] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.635231 25826 raft_consensus.cc:385] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.638273 25826 raft_consensus.cc:740] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.638774 25827 tablet_bootstrap.cc:654] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.639192 25826 consensus_queue.cc:260] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.641953 25826 ts_tablet_manager.cc:1434] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:32.643254 25826 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.643877 25858 raft_consensus.cc:493] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.644520 25858 raft_consensus.cc:515] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.648029 25858 leader_election.cc:290] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.647931 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.647931 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.649780 25785 raft_consensus.cc:2468] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
W20260709 13:07:32.651391 25674 leader_election.cc:343] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.652144 25677 leader_election.cc:304] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:32.654165 25858 raft_consensus.cc:2804] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.654660 25858 raft_consensus.cc:493] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.655107 25858 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.646821 25529 catalog_manager.cc:5697] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.671427 25825 tablet_bootstrap.cc:654] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.675565 25826 tablet_bootstrap.cc:654] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.691651 25530 catalog_manager.cc:5697] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 1 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.694025 25825 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.694698 25825 ts_tablet_manager.cc:1403] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.004s
I20260709 13:07:32.696113 25858 raft_consensus.cc:515] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.699462 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
W20260709 13:07:32.700901 25674 leader_election.cc:343] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.701485 25825 raft_consensus.cc:359] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.702332 25825 raft_consensus.cc:385] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.702646 25825 raft_consensus.cc:740] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.703814 25827 tablet_bootstrap.cc:492] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:32.704368 25827 ts_tablet_manager.cc:1403] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.082s	user 0.022s	sys 0.018s
I20260709 13:07:32.707146 25827 raft_consensus.cc:359] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.707828 25827 raft_consensus.cc:385] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.708108 25827 raft_consensus.cc:740] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.708464 25825 consensus_queue.cc:260] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.708829 25827 consensus_queue.cc:260] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.710978 25825 ts_tablet_manager.cc:1434] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:07:32.713240 25827 ts_tablet_manager.cc:1434] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:07:32.715077 25858 leader_election.cc:290] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.715721 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:32.716435 25785 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.717260 25825 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.718385 25826 tablet_bootstrap.cc:492] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.718938 25826 ts_tablet_manager.cc:1403] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.076s	user 0.032s	sys 0.013s
I20260709 13:07:32.721480 25826 raft_consensus.cc:359] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.722118 25826 raft_consensus.cc:385] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.722462 25826 raft_consensus.cc:740] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.723623 25826 consensus_queue.cc:260] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.724771 25825 tablet_bootstrap.cc:654] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.724872 25785 raft_consensus.cc:2468] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:32.726073 25826 ts_tablet_manager.cc:1434] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:07:32.726361 25677 leader_election.cc:304] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:32.727380 25826 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.727775 25858 raft_consensus.cc:2804] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.728379 25858 raft_consensus.cc:697] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:32.729331 25858 consensus_queue.cc:237] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.735030 25826 tablet_bootstrap.cc:654] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.736492 25832 raft_consensus.cc:493] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.737146 25832 raft_consensus.cc:515] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.741235 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.743425 25832 leader_election.cc:290] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.745316 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
W20260709 13:07:32.745376 25601 leader_election.cc:343] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.749641 25785 raft_consensus.cc:2468] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:32.751528 25602 leader_election.cc:304] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:32.753335 25832 raft_consensus.cc:2804] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.753882 25832 raft_consensus.cc:493] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.754355 25832 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.766618 25832 raft_consensus.cc:515] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.771034 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
W20260709 13:07:32.772624 25601 leader_election.cc:343] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:32.775343 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:32.776273 25785 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.777084 25533 catalog_manager.cc:5697] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.785144 25832 leader_election.cc:290] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.790298 25785 raft_consensus.cc:2468] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:32.792681 25602 leader_election.cc:304] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:32.794624 25832 raft_consensus.cc:2804] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.796566 25832 raft_consensus.cc:697] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:32.797950 25832 consensus_queue.cc:237] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.811349 25825 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.811690 25834 raft_consensus.cc:493] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.812104 25825 ts_tablet_manager.cc:1403] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.095s	user 0.053s	sys 0.032s
I20260709 13:07:32.812275 25834 raft_consensus.cc:515] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.814148 25826 tablet_bootstrap.cc:492] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.814567 25834 leader_election.cc:290] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187)
I20260709 13:07:32.814749 25826 ts_tablet_manager.cc:1403] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.088s	user 0.030s	sys 0.053s
I20260709 13:07:32.815608 25825 raft_consensus.cc:359] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.816399 25825 raft_consensus.cc:385] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.816726 25825 raft_consensus.cc:740] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.817606 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.817049 25533 catalog_manager.cc:5697] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.817766 25825 consensus_queue.cc:260] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.818341 25636 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.818378 25826 raft_consensus.cc:359] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.819340 25826 raft_consensus.cc:385] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.819722 25826 raft_consensus.cc:740] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.819875 25748 leader_election.cc:304] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.821022 25834 raft_consensus.cc:2804] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.820559 25826 consensus_queue.cc:260] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.821476 25834 raft_consensus.cc:493] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.821908 25834 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.823347 25826 ts_tablet_manager.cc:1434] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:32.824296 25825 ts_tablet_manager.cc:1434] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:07:32.824736 25826 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:32.820700 25832 raft_consensus.cc:493] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.825906 25825 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.826121 25832 raft_consensus.cc:515] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.829345 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.816418 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.832418 25711 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.832620 25826 tablet_bootstrap.cc:654] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.833668 25825 tablet_bootstrap.cc:654] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.833937 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.834842 25785 raft_consensus.cc:2393] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 466dd86270c5461ca8e12b502390e16a in current term 1: Already voted for candidate 79c6af590cca4da7b26cbcca34e2cdbe in this term.
I20260709 13:07:32.837478 25832 leader_election.cc:290] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.840349 25602 leader_election.cc:304] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a; no voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:32.845515 25832 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.844231 25834 raft_consensus.cc:515] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.859345 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:32.860028 25711 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.861488 25832 raft_consensus.cc:2749] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:32.868791 25711 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.869724 25834 leader_election.cc:290] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187)
I20260709 13:07:32.870857 25750 leader_election.cc:304] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.872712 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:32.873413 25834 raft_consensus.cc:2804] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.873446 25636 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.874461 25825 tablet_bootstrap.cc:492] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.874991 25825 ts_tablet_manager.cc:1403] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.050s	user 0.026s	sys 0.020s
I20260709 13:07:32.875061 25834 raft_consensus.cc:697] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:32.876202 25834 consensus_queue.cc:237] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:32.877347 25826 tablet_bootstrap.cc:492] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:32.878144 25825 raft_consensus.cc:359] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.878890 25825 raft_consensus.cc:385] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.879154 25825 raft_consensus.cc:740] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.878161 25826 ts_tablet_manager.cc:1403] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.017s
I20260709 13:07:32.880036 25825 consensus_queue.cc:260] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.882462 25826 raft_consensus.cc:359] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.883325 25826 raft_consensus.cc:385] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.883652 25826 raft_consensus.cc:740] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.884490 25826 consensus_queue.cc:260] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.884737 25636 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.887169 25826 ts_tablet_manager.cc:1434] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:32.889181 25825 ts_tablet_manager.cc:1434] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:32.890455 25825 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:32.896665 25825 tablet_bootstrap.cc:654] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.899757 25529 catalog_manager.cc:5697] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 1 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.902103 25825 tablet_bootstrap.cc:492] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:32.902671 25825 ts_tablet_manager.cc:1403] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:07:32.905568 25825 raft_consensus.cc:359] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.906276 25825 raft_consensus.cc:385] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.906555 25825 raft_consensus.cc:740] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.907265 25825 consensus_queue.cc:260] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.909355 25825 ts_tablet_manager.cc:1434] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:07:32.942708 25834 raft_consensus.cc:493] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.943306 25834 raft_consensus.cc:515] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.945243 25834 leader_election.cc:290] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.945998 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f1414dc93b84fa49258bd56a9334f65" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.946343 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f1414dc93b84fa49258bd56a9334f65" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:32.946866 25636 raft_consensus.cc:2468] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.947007 25711 raft_consensus.cc:2468] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:32.948125 25748 leader_election.cc:304] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.949007 25834 raft_consensus.cc:2804] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.949366 25834 raft_consensus.cc:493] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.949638 25834 raft_consensus.cc:3060] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.954639 25834 raft_consensus.cc:515] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.956338 25834 leader_election.cc:290] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:32.957597 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f1414dc93b84fa49258bd56a9334f65" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:32.957669 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f1414dc93b84fa49258bd56a9334f65" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:32.958220 25711 raft_consensus.cc:3060] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.958514 25636 raft_consensus.cc:3060] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.963673 25711 raft_consensus.cc:2468] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.963673 25636 raft_consensus.cc:2468] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:32.965000 25750 leader_election.cc:304] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:32.965778 25834 raft_consensus.cc:2804] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.966259 25834 raft_consensus.cc:697] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:32.966889 25834 consensus_queue.cc:237] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.974576 25529 catalog_manager.cc:5697] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 1 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.991264 25858 raft_consensus.cc:493] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.991953 25858 raft_consensus.cc:515] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:32.994019 25858 leader_election.cc:290] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:32.995018 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:32.995187 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:32.995734 25636 raft_consensus.cc:2468] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:32.995843 25785 raft_consensus.cc:2468] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:32.996837 25674 leader_election.cc:304] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:32.997711 25858 raft_consensus.cc:2804] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:32.998121 25858 raft_consensus.cc:493] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.998481 25858 raft_consensus.cc:3060] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.004307 25858 raft_consensus.cc:515] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.006273 25858 leader_election.cc:290] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.007112 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:33.007520 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175d87f9e1d04e73bf1464b3070e7370" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.007972 25636 raft_consensus.cc:3060] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.008196 25785 raft_consensus.cc:3060] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.010380 25832 consensus_queue.cc:1048] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.016574 25832 consensus_queue.cc:1048] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.021683 25636 raft_consensus.cc:2468] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:33.022008 25785 raft_consensus.cc:2468] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:33.023260 25674 leader_election.cc:304] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:33.024535 25858 raft_consensus.cc:2804] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:33.025295 25858 raft_consensus.cc:697] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:33.026147 25858 consensus_queue.cc:237] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.039018 25530 catalog_manager.cc:5697] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.044826 25834 consensus_queue.cc:1048] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.055969 25711 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.062358 25710 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.066294 25832 raft_consensus.cc:493] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.066892 25832 raft_consensus.cc:515] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.069051 25832 leader_election.cc:290] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.070565 25889 consensus_queue.cc:1048] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.071683 25834 consensus_queue.cc:1048] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.071964 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:33.072808 25784 raft_consensus.cc:2393] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 466dd86270c5461ca8e12b502390e16a in current term 1: Already voted for candidate 79c6af590cca4da7b26cbcca34e2cdbe in this term.
I20260709 13:07:33.070706 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:33.076568 25601 leader_election.cc:304] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a; no voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:33.078303 25832 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.086032 25832 raft_consensus.cc:2749] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:33.095170 25863 consensus_queue.cc:1048] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.104561 25889 consensus_queue.cc:1048] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.123461 25863 consensus_queue.cc:1048] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.156479 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
mode: GRACEFUL
new_leader_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:44754
I20260709 13:07:33.157076 25784 raft_consensus.cc:606] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Received request to transfer leadership to TS 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:33.173318 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
mode: GRACEFUL
new_leader_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:07:33.173868 25709 raft_consensus.cc:606] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Received request to transfer leadership to TS 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:33.179327 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
mode: GRACEFUL
new_leader_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:44754
I20260709 13:07:33.179885 25784 raft_consensus.cc:606] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Received request to transfer leadership to TS 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:33.190130 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
mode: GRACEFUL
new_leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:44754
I20260709 13:07:33.190847 25784 raft_consensus.cc:606] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Received request to transfer leadership to TS 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:33.204768 25635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
mode: GRACEFUL
new_leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:54550
I20260709 13:07:33.205420 25635 raft_consensus.cc:606] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Received request to transfer leadership to TS 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:33.259174 25709 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.265380 25832 consensus_queue.cc:1048] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.272678 25856 raft_consensus.cc:993] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: : Instructing follower 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:33.273238 25854 raft_consensus.cc:1081] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Signalling peer 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:33.275447 25710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:35824
I20260709 13:07:33.276126 25710 raft_consensus.cc:493] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.276592 25710 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.281801 25856 consensus_queue.cc:1048] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.286058 25710 raft_consensus.cc:515] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.290273 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:33.291656 25635 raft_consensus.cc:3055] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:33.292079 25635 raft_consensus.cc:740] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:33.292363 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fb62cb171046f1a45df62a2c4efcff" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.293192 25784 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.293195 25635 consensus_queue.cc:260] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.294379 25710 leader_election.cc:290] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.294732 25635 raft_consensus.cc:3060] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.301262 25784 raft_consensus.cc:2468] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:33.302536 25677 leader_election.cc:304] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:33.303498 25858 raft_consensus.cc:2804] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.304045 25858 raft_consensus.cc:697] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:33.304498 25635 raft_consensus.cc:2468] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:33.304971 25858 consensus_queue.cc:237] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.308245 25893 consensus_queue.cc:1048] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.314775 25529 catalog_manager.cc:5697] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 1 to 2, leader changed from 466dd86270c5461ca8e12b502390e16a (127.13.163.193) to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.319950 25893 consensus_queue.cc:1048] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.333942 25889 consensus_queue.cc:1048] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.346621 25840 consensus_queue.cc:1048] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.356598 25834 raft_consensus.cc:993] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: : Instructing follower 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.357112 25889 raft_consensus.cc:1081] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Signalling peer 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.359129 25635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:38028
I20260709 13:07:33.359786 25635 raft_consensus.cc:493] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.360148 25635 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.365702 25635 raft_consensus.cc:515] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:33.367901 25635 leader_election.cc:290] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Requested vote from peers fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:33.368642 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.369015 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbb0f118713403e92a23cf7f1d71eda" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:33.369617 25710 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.370302 25784 raft_consensus.cc:3055] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:33.370743 25784 raft_consensus.cc:740] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:33.371536 25784 consensus_queue.cc:260] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:33.372874 25784 raft_consensus.cc:3060] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.377116 25710 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.378481 25601 leader_election.cc:304] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:33.379452 25856 raft_consensus.cc:2804] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.379995 25856 raft_consensus.cc:697] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:33.380239 25784 raft_consensus.cc:2468] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.380995 25856 consensus_queue.cc:237] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } }
I20260709 13:07:33.389215 25529 catalog_manager.cc:5697] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 1 to 2, leader changed from fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195) to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 2 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.432777 25889 consensus_queue.cc:1048] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.449117 25834 consensus_queue.cc:1048] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.516700 25893 consensus_queue.cc:1048] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.528512 25863 consensus_queue.cc:1048] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:33.536290 25863 raft_consensus.cc:993] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: : Instructing follower 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.536772 25893 raft_consensus.cc:1081] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Signalling peer 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.538369 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:38032
I20260709 13:07:33.538868 25636 raft_consensus.cc:493] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.539187 25636 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.545513 25636 raft_consensus.cc:515] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.547236 25636 leader_election.cc:290] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.548127 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:33.548276 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daae2603a8ad48b4ad155c7971a07739" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.548794 25710 raft_consensus.cc:3055] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:33.548945 25784 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.549172 25710 raft_consensus.cc:740] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:33.550040 25710 consensus_queue.cc:260] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.551017 25710 raft_consensus.cc:3060] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.554272 25784 raft_consensus.cc:2468] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.555444 25602 leader_election.cc:304] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:33.556134 25710 raft_consensus.cc:2468] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.556160 25854 raft_consensus.cc:2804] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.556878 25854 raft_consensus.cc:697] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:33.557636 25854 consensus_queue.cc:237] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.564069 25533 catalog_manager.cc:5697] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 1 to 2, leader changed from 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194) to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 2 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.613520 25840 raft_consensus.cc:993] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: : Instructing follower 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:33.614040 25840 raft_consensus.cc:1081] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Signalling peer 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:33.615841 25710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:35830
I20260709 13:07:33.616375 25710 raft_consensus.cc:493] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.616801 25710 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.622380 25710 raft_consensus.cc:515] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.624178 25710 leader_election.cc:290] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.625051 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:33.625385 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bdbf339b8f941baaffa2d12334932f0" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.625787 25636 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.626039 25784 raft_consensus.cc:3055] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:33.626430 25784 raft_consensus.cc:740] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:33.627359 25784 consensus_queue.cc:260] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.628736 25784 raft_consensus.cc:3060] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.633663 25636 raft_consensus.cc:2468] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:33.635149 25674 leader_election.cc:304] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:33.635879 25893 raft_consensus.cc:2804] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.636303 25893 raft_consensus.cc:697] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:33.636247 25784 raft_consensus.cc:2468] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:33.637318 25893 consensus_queue.cc:237] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.644395 25533 catalog_manager.cc:5697] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 1 to 2, leader changed from fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195) to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.692909 25834 raft_consensus.cc:993] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: : Instructing follower 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.693408 25840 raft_consensus.cc:1081] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Signalling peer 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:33.694931 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:38028
I20260709 13:07:33.695529 25636 raft_consensus.cc:493] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.695925 25636 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.701135 25636 raft_consensus.cc:515] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.702790 25636 leader_election.cc:290] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:33.703653 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:33.703856 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c2b59aab64796be18c4dc3adaca06" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:33.704311 25710 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.704504 25784 raft_consensus.cc:3055] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:33.705124 25784 raft_consensus.cc:740] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:33.705876 25784 consensus_queue.cc:260] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.706854 25784 raft_consensus.cc:3060] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:33.709807 25710 raft_consensus.cc:2468] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.711067 25601 leader_election.cc:304] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:33.711828 25854 raft_consensus.cc:2804] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.712271 25854 raft_consensus.cc:697] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:33.712266 25784 raft_consensus.cc:2468] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:33.712785 25785 raft_consensus.cc:1275] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:33.713809 25854 consensus_queue.cc:237] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:33.714381 25893 consensus_queue.cc:1048] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:33.721572 25533 catalog_manager.cc:5697] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 1 to 2, leader changed from fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195) to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 2 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.722525 25636 raft_consensus.cc:1275] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:33.724720 25863 consensus_queue.cc:1048] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:33.873034 25710 raft_consensus.cc:1275] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:33.874506 25854 consensus_queue.cc:1048] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:33.883320 25785 raft_consensus.cc:1275] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:33.884907 25910 consensus_queue.cc:1048] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:34.042696 25785 raft_consensus.cc:1275] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.044364 25863 consensus_queue.cc:1048] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.052410 25636 raft_consensus.cc:1275] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.054158 25893 consensus_queue.cc:1048] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.077100 25710 raft_consensus.cc:1275] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.078722 25910 consensus_queue.cc:1048] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.086302 25785 raft_consensus.cc:1275] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.087754 25854 consensus_queue.cc:1048] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.175437 25785 raft_consensus.cc:1275] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.176884 25854 consensus_queue.cc:1048] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.186368 25710 raft_consensus.cc:1275] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.188041 25854 consensus_queue.cc:1048] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:36.243296 25533 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51066:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:36.246120 25533 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:36.305166 25622 tablet_service.cc:1511] Processing CreateTablet for tablet ab53fc1ca0f149dca173e9867389ae94 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.306743 25698 tablet_service.cc:1511] Processing CreateTablet for tablet ab53fc1ca0f149dca173e9867389ae94 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.307220 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab53fc1ca0f149dca173e9867389ae94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.308440 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab53fc1ca0f149dca173e9867389ae94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.314221 25767 tablet_service.cc:1511] Processing CreateTablet for tablet ab53fc1ca0f149dca173e9867389ae94 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.314851 25621 tablet_service.cc:1511] Processing CreateTablet for tablet c8443cc0e7ac464187d88297b95ac54f (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.316062 25767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab53fc1ca0f149dca173e9867389ae94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.316442 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8443cc0e7ac464187d88297b95ac54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.318217 25618 tablet_service.cc:1511] Processing CreateTablet for tablet 16701afc3eee4f9abc7d317abd0c5e3a (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.312609 25771 tablet_service.cc:1511] Processing CreateTablet for tablet 16701afc3eee4f9abc7d317abd0c5e3a (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.320242 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16701afc3eee4f9abc7d317abd0c5e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.321147 25694 tablet_service.cc:1511] Processing CreateTablet for tablet 16701afc3eee4f9abc7d317abd0c5e3a (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.322783 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16701afc3eee4f9abc7d317abd0c5e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.326587 25697 tablet_service.cc:1511] Processing CreateTablet for tablet c8443cc0e7ac464187d88297b95ac54f (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.333461 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8443cc0e7ac464187d88297b95ac54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.337302 25618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16701afc3eee4f9abc7d317abd0c5e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.354278 25926 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.360797 25768 tablet_service.cc:1511] Processing CreateTablet for tablet c8443cc0e7ac464187d88297b95ac54f (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.362421 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8443cc0e7ac464187d88297b95ac54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.364717 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.365319 25926 tablet_bootstrap.cc:654] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.366492 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.375679 25693 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.376611 25767 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.377216 25693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.377969 25767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4c41ba20d54b4bb7a76a6c68f118e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.390646 25927 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:36.406024 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 101e50fde3eb4118a8cd3c020c7933e1 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.407706 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101e50fde3eb4118a8cd3c020c7933e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.409991 25622 tablet_service.cc:1511] Processing CreateTablet for tablet 101e50fde3eb4118a8cd3c020c7933e1 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.411430 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101e50fde3eb4118a8cd3c020c7933e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.447309 25929 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:36.451774 25927 tablet_bootstrap.cc:654] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.454001 25929 tablet_bootstrap.cc:654] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.459614 25768 tablet_service.cc:1511] Processing CreateTablet for tablet 101e50fde3eb4118a8cd3c020c7933e1 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.465819 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101e50fde3eb4118a8cd3c020c7933e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.466430 25926 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.467020 25926 ts_tablet_manager.cc:1403] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.113s	user 0.023s	sys 0.052s
I20260709 13:07:36.462433 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 3c252c629b6d480abbec264834467a23 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:36.469735 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c252c629b6d480abbec264834467a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.470633 25926 raft_consensus.cc:359] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.471688 25926 raft_consensus.cc:385] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.471997 25926 raft_consensus.cc:740] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.476356 25926 consensus_queue.cc:260] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.477890 25840 raft_consensus.cc:493] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.478404 25840 raft_consensus.cc:515] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.479317 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 3c252c629b6d480abbec264834467a23 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:36.480880 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c252c629b6d480abbec264834467a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.483172 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:36.485080 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:36.485226 25767 tablet_service.cc:1511] Processing CreateTablet for tablet 3c252c629b6d480abbec264834467a23 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
W20260709 13:07:36.487439 25748 leader_election.cc:343] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.488229 25840 leader_election.cc:290] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
W20260709 13:07:36.488827 25750 leader_election.cc:343] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.489367 25750 leader_election.cc:304] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:36.488297 25767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c252c629b6d480abbec264834467a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.491245 25840 raft_consensus.cc:2749] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.491726 25622 tablet_service.cc:1511] Processing CreateTablet for tablet 1b274557ca07475583c64414a80db13e (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.497397 25929 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:36.498070 25929 ts_tablet_manager.cc:1403] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.007s
I20260709 13:07:36.498311 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b274557ca07475583c64414a80db13e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:36.499815 25582 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:36.501250 25693 tablet_service.cc:1511] Processing CreateTablet for tablet 1b274557ca07475583c64414a80db13e (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.502898 25693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b274557ca07475583c64414a80db13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.502923 25926 ts_tablet_manager.cc:1434] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:07:36.518580 25926 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.522460 25927 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:36.523084 25927 ts_tablet_manager.cc:1403] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.133s	user 0.009s	sys 0.007s
I20260709 13:07:36.524041 25929 raft_consensus.cc:359] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.524915 25929 raft_consensus.cc:385] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.525244 25929 raft_consensus.cc:740] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.526007 25929 consensus_queue.cc:260] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.526336 25927 raft_consensus.cc:359] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.527119 25927 raft_consensus.cc:385] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.527436 25927 raft_consensus.cc:740] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.528273 25927 consensus_queue.cc:260] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.529913 25926 tablet_bootstrap.cc:654] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:36.533313 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 45462.5us
I20260709 13:07:36.536862 25768 tablet_service.cc:1511] Processing CreateTablet for tablet 1b274557ca07475583c64414a80db13e (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.538527 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b274557ca07475583c64414a80db13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.538828 25927 ts_tablet_manager.cc:1434] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:07:36.540257 25622 tablet_service.cc:1511] Processing CreateTablet for tablet c381f41065ab4d7fbfb26b80cc3d5287 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.541676 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c381f41065ab4d7fbfb26b80cc3d5287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.544188 25926 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.544943 25926 ts_tablet_manager.cc:1403] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.005s
I20260709 13:07:36.548228 25926 raft_consensus.cc:359] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.549165 25926 raft_consensus.cc:385] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.549556 25926 raft_consensus.cc:740] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.550483 25926 consensus_queue.cc:260] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.551232 25694 tablet_service.cc:1511] Processing CreateTablet for tablet c381f41065ab4d7fbfb26b80cc3d5287 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.556087 25926 ts_tablet_manager.cc:1434] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:07:36.557407 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 9080c81947294f0b82051bbe4b2e1670 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.557843 25926 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.558825 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9080c81947294f0b82051bbe4b2e1670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.564457 25929 ts_tablet_manager.cc:1434] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 13:07:36.566188 25926 tablet_bootstrap.cc:654] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.570364 25771 tablet_service.cc:1511] Processing CreateTablet for tablet 9080c81947294f0b82051bbe4b2e1670 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.571696 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9080c81947294f0b82051bbe4b2e1670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.573994 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 9080c81947294f0b82051bbe4b2e1670 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.573890 25767 tablet_service.cc:1511] Processing CreateTablet for tablet c381f41065ab4d7fbfb26b80cc3d5287 (DEFAULT_TABLE table=soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.582243 25767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c381f41065ab4d7fbfb26b80cc3d5287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.585381 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c381f41065ab4d7fbfb26b80cc3d5287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.591923 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9080c81947294f0b82051bbe4b2e1670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.592020 25927 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:36.598837 25929 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:36.612478 25929 tablet_bootstrap.cc:654] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.614765 25927 tablet_bootstrap.cc:654] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.613327 25926 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.615626 25926 ts_tablet_manager.cc:1403] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.058s	user 0.006s	sys 0.010s
I20260709 13:07:36.620338 25926 raft_consensus.cc:359] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.625005 25926 raft_consensus.cc:385] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.625588 25926 raft_consensus.cc:740] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.628595 25926 consensus_queue.cc:260] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.639977 25927 tablet_bootstrap.cc:492] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:36.640230 25926 ts_tablet_manager.cc:1434] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:07:36.640527 25927 ts_tablet_manager.cc:1403] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.011s
I20260709 13:07:36.641538 25926 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.643900 25927 raft_consensus.cc:359] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.644655 25927 raft_consensus.cc:385] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.644968 25927 raft_consensus.cc:740] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.645718 25927 consensus_queue.cc:260] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.648109 25927 ts_tablet_manager.cc:1434] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:07:36.649408 25929 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:36.650205 25929 ts_tablet_manager.cc:1403] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.018s
I20260709 13:07:36.649488 25926 tablet_bootstrap.cc:654] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.650877 25927 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:36.653759 25929 raft_consensus.cc:359] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.654838 25929 raft_consensus.cc:385] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.655310 25929 raft_consensus.cc:740] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.656317 25929 consensus_queue.cc:260] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.659359 25929 ts_tablet_manager.cc:1434] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:36.659487 25927 tablet_bootstrap.cc:654] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.660535 25929 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:36.671074 25929 tablet_bootstrap.cc:654] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.674633 25926 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.675362 25926 ts_tablet_manager.cc:1403] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.034s	user 0.026s	sys 0.003s
I20260709 13:07:36.679498 25926 raft_consensus.cc:359] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.680673 25926 raft_consensus.cc:385] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.681154 25926 raft_consensus.cc:740] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.682410 25926 consensus_queue.cc:260] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.686538 25926 ts_tablet_manager.cc:1434] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.008s
I20260709 13:07:36.687889 25926 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.696226 25926 tablet_bootstrap.cc:654] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.734375 25927 tablet_bootstrap.cc:492] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:36.734956 25927 ts_tablet_manager.cc:1403] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.084s	user 0.031s	sys 0.037s
I20260709 13:07:36.735056 25926 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.735803 25926 ts_tablet_manager.cc:1403] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.048s	user 0.027s	sys 0.018s
I20260709 13:07:36.738397 25927 raft_consensus.cc:359] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.739076 25927 raft_consensus.cc:385] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.738751 25926 raft_consensus.cc:359] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.739424 25927 raft_consensus.cc:740] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.739586 25926 raft_consensus.cc:385] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.740027 25926 raft_consensus.cc:740] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.740362 25927 consensus_queue.cc:260] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.740989 25926 consensus_queue.cc:260] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.742183 25929 tablet_bootstrap.cc:492] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:36.742738 25929 ts_tablet_manager.cc:1403] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.082s	user 0.021s	sys 0.026s
I20260709 13:07:36.743175 25927 ts_tablet_manager.cc:1434] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:36.745102 25926 ts_tablet_manager.cc:1434] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:07:36.746320 25926 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.752995 25929 raft_consensus.cc:359] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.753727 25927 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:36.754053 25926 tablet_bootstrap.cc:654] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.753778 25929 raft_consensus.cc:385] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.758067 25929 raft_consensus.cc:740] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.758998 25929 consensus_queue.cc:260] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.760528 25926 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.761133 25926 ts_tablet_manager.cc:1403] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:07:36.761319 25927 tablet_bootstrap.cc:654] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.761564 25929 ts_tablet_manager.cc:1434] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 13:07:36.762641 25929 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:36.764626 25926 raft_consensus.cc:359] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.765537 25926 raft_consensus.cc:385] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.765851 25926 raft_consensus.cc:740] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.766656 25926 consensus_queue.cc:260] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.769593 25926 ts_tablet_manager.cc:1434] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:36.770771 25926 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.771394 25929 tablet_bootstrap.cc:654] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.778481 25926 tablet_bootstrap.cc:654] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.781993 25932 raft_consensus.cc:493] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.782856 25932 raft_consensus.cc:515] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.787407 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
W20260709 13:07:36.790230 25748 leader_election.cc:343] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.791728 25863 raft_consensus.cc:493] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.792598 25863 raft_consensus.cc:515] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.803568 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:36.804670 25785 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:36.807183 25677 leader_election.cc:304] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:36.810803 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:36.811905 25636 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:36.818378 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
W20260709 13:07:36.820712 25750 leader_election.cc:343] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.821483 25750 leader_election.cc:304] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:36.822768 25840 raft_consensus.cc:2749] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.823899 25932 leader_election.cc:290] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:36.829720 25863 leader_election.cc:290] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:36.838929 25863 raft_consensus.cc:2804] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.841159 25863 raft_consensus.cc:493] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.842718 25863 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.840241 25932 raft_consensus.cc:493] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.857374 25932 raft_consensus.cc:515] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.862074 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
W20260709 13:07:36.864384 25748 leader_election.cc:343] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.870584 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
W20260709 13:07:36.872829 25750 leader_election.cc:343] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.873564 25750 leader_election.cc:304] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:36.874740 25840 raft_consensus.cc:2749] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.883028 25932 leader_election.cc:290] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
W20260709 13:07:36.887213 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:33817, user_credentials={real_user=slave}} blocked reactor thread for 80333.5us
W20260709 13:07:36.892032 25674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:43187, user_credentials={real_user=slave}} blocked reactor thread for 70747.8us
I20260709 13:07:36.894627 25948 raft_consensus.cc:493] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.895358 25948 raft_consensus.cc:515] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.899725 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:36.898466 25948 leader_election.cc:290] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:36.901014 25636 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:36.903227 25674 leader_election.cc:304] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:36.903905 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:36.904695 25948 raft_consensus.cc:2804] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.904978 25785 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:36.905337 25948 raft_consensus.cc:493] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.905938 25948 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.920446 25840 raft_consensus.cc:493] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.921309 25840 raft_consensus.cc:515] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.926007 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:36.928172 25840 leader_election.cc:290] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
W20260709 13:07:36.929642 25748 leader_election.cc:343] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.930924 25948 raft_consensus.cc:515] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.933981 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:36.943960 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:36.949272 25636 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:36.956739 25750 leader_election.cc:343] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.957621 25750 leader_election.cc:304] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:36.960047 25840 raft_consensus.cc:2749] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.964072 25926 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.964715 25926 ts_tablet_manager.cc:1403] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.194s	user 0.072s	sys 0.115s
I20260709 13:07:36.964635 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:36.965550 25785 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.965621 25927 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:36.966295 25927 ts_tablet_manager.cc:1403] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.213s	user 0.069s	sys 0.133s
I20260709 13:07:36.966846 25636 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:36.968480 25674 leader_election.cc:304] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:36.968354 25926 raft_consensus.cc:359] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.969360 25926 raft_consensus.cc:385] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.969738 25926 raft_consensus.cc:740] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.969856 25947 raft_consensus.cc:2804] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.970842 25947 raft_consensus.cc:697] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:36.970651 25926 consensus_queue.cc:260] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.971989 25947 consensus_queue.cc:237] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.972644 25863 raft_consensus.cc:515] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.973287 25926 ts_tablet_manager.cc:1434] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:07:36.974385 25926 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.969583 25927 raft_consensus.cc:359] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.976398 25927 raft_consensus.cc:385] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.976792 25927 raft_consensus.cc:740] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.977281 25785 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:36.978237 25840 raft_consensus.cc:493] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.978835 25840 raft_consensus.cc:515] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.979091 25927 consensus_queue.cc:260] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.981231 25840 leader_election.cc:290] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:36.981312 25926 tablet_bootstrap.cc:654] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.983759 25927 ts_tablet_manager.cc:1434] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:07:36.988497 25927 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:36.989046 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
W20260709 13:07:36.990854 25748 leader_election.cc:343] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.992828 25854 raft_consensus.cc:493] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.993049 25926 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:36.993723 25926 ts_tablet_manager.cc:1403] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:07:36.993472 25854 raft_consensus.cc:515] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.993767 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:36.995332 25863 leader_election.cc:290] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:36.996322 25854 leader_election.cc:290] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:36.993950 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:36.996956 25926 raft_consensus.cc:359] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:36.997578 25636 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.997787 25926 raft_consensus.cc:385] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.997851 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:36.998283 25926 raft_consensus.cc:740] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
W20260709 13:07:36.996347 25750 leader_election.cc:343] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.999109 25750 leader_election.cc:304] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:36.999011 25926 consensus_queue.cc:260] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.000450 25949 raft_consensus.cc:2749] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.002678 25926 ts_tablet_manager.cc:1434] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:37.003767 25926 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: Bootstrap starting.
I20260709 13:07:36.976878 25948 leader_election.cc:290] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.006836 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.007797 25783 raft_consensus.cc:2468] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:36.998627 25784 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.009106 25636 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.010447 25926 tablet_bootstrap.cc:654] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.013737 25674 leader_election.cc:304] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.015136 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:37.015868 25602 leader_election.cc:304] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.015913 25710 raft_consensus.cc:2468] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.016986 25854 raft_consensus.cc:2804] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.017364 25854 raft_consensus.cc:493] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.017684 25854 raft_consensus.cc:3060] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.020159 25948 raft_consensus.cc:2804] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.020265 25784 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.020891 25948 raft_consensus.cc:697] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:37.022035 25948 consensus_queue.cc:237] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.025491 25926 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: No bootstrap required, opened a new log
I20260709 13:07:37.026050 25926 ts_tablet_manager.cc:1403] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.005s
I20260709 13:07:37.026981 25929 tablet_bootstrap.cc:492] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.027539 25929 ts_tablet_manager.cc:1403] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.265s	user 0.027s	sys 0.047s
I20260709 13:07:37.027208 25854 raft_consensus.cc:515] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.028954 25926 raft_consensus.cc:359] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.015189 25927 tablet_bootstrap.cc:654] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.029654 25926 raft_consensus.cc:385] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.030236 25926 raft_consensus.cc:740] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.031154 25926 consensus_queue.cc:260] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.031635 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.032593 25784 raft_consensus.cc:3060] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.033631 25926 ts_tablet_manager.cc:1434] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:07:37.036899 25840 raft_consensus.cc:493] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.037675 25840 raft_consensus.cc:515] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.040100 25784 raft_consensus.cc:2468] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.041461 25602 leader_election.cc:304] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.042127 25840 leader_election.cc:290] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:37.043115 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:37.043677 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
W20260709 13:07:37.045019 25748 leader_election.cc:343] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.046609 25750 leader_election.cc:343] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.049027 25854 leader_election.cc:290] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.051781 25854 raft_consensus.cc:2804] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.052381 25854 raft_consensus.cc:697] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:37.053408 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab53fc1ca0f149dca173e9867389ae94" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.063961 25710 raft_consensus.cc:3060] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.055678 25750 leader_election.cc:304] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:37.065582 25840 raft_consensus.cc:2749] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.062687 25929 raft_consensus.cc:359] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.053617 25854 consensus_queue.cc:237] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.075832 25533 catalog_manager.cc:5697] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.078253 25533 catalog_manager.cc:5697] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.080330 25929 raft_consensus.cc:385] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.080930 25929 raft_consensus.cc:740] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.081663 25956 raft_consensus.cc:493] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.082195 25956 raft_consensus.cc:515] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.083398 25929 consensus_queue.cc:260] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.084650 25956 leader_election.cc:290] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.086321 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:37.087086 25709 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.089361 25929 ts_tablet_manager.cc:1434] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.004s
I20260709 13:07:37.092132 25601 leader_election.cc:304] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.094389 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.095139 25785 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.097110 25956 raft_consensus.cc:2804] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.097594 25956 raft_consensus.cc:493] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.097918 25956 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.102429 25929 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:37.107952 25927 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:37.107923 25710 raft_consensus.cc:2468] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.108737 25927 ts_tablet_manager.cc:1403] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.124s	user 0.043s	sys 0.039s
I20260709 13:07:37.112707 25956 raft_consensus.cc:515] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.112373 25529 catalog_manager.cc:5697] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.112154 25927 raft_consensus.cc:359] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.114288 25927 raft_consensus.cc:385] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.114589 25927 raft_consensus.cc:740] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.117321 25956 leader_election.cc:290] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.118378 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.119047 25784 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.119171 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.119948 25710 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.115309 25927 consensus_queue.cc:260] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.126695 25784 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.127861 25927 ts_tablet_manager.cc:1434] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:07:37.128113 25602 leader_election.cc:304] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.129227 25955 raft_consensus.cc:2804] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.129449 25710 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.129724 25955 raft_consensus.cc:697] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:37.130703 25955 consensus_queue.cc:237] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.133208 25929 tablet_bootstrap.cc:654] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.129524 25927 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:37.147930 25929 tablet_bootstrap.cc:492] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.148451 25927 tablet_bootstrap.cc:654] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.148553 25929 ts_tablet_manager.cc:1403] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.006s
I20260709 13:07:37.152194 25929 raft_consensus.cc:359] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.152981 25929 raft_consensus.cc:385] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.153298 25929 raft_consensus.cc:740] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.154310 25929 consensus_queue.cc:260] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.158034 25929 ts_tablet_manager.cc:1434] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:07:37.159216 25929 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:37.159663 25927 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:37.160215 25927 ts_tablet_manager.cc:1403] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.004s
I20260709 13:07:37.161540 25533 catalog_manager.cc:5697] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.163270 25927 raft_consensus.cc:359] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.164085 25927 raft_consensus.cc:385] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.164383 25927 raft_consensus.cc:740] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.165189 25927 consensus_queue.cc:260] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.167164 25929 tablet_bootstrap.cc:654] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.167677 25927 ts_tablet_manager.cc:1434] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:07:37.168630 25927 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:37.172889 25929 tablet_bootstrap.cc:492] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.173516 25929 ts_tablet_manager.cc:1403] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.007s
I20260709 13:07:37.175164 25927 tablet_bootstrap.cc:654] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.176476 25929 raft_consensus.cc:359] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.177214 25929 raft_consensus.cc:385] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.177461 25929 raft_consensus.cc:740] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.178221 25929 consensus_queue.cc:260] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.180364 25929 ts_tablet_manager.cc:1434] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:37.181406 25929 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:37.185441 25927 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:37.186022 25927 ts_tablet_manager.cc:1403] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:07:37.187412 25929 tablet_bootstrap.cc:654] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.188889 25927 raft_consensus.cc:359] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.189654 25927 raft_consensus.cc:385] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.189901 25927 raft_consensus.cc:740] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.190560 25927 consensus_queue.cc:260] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.192804 25927 ts_tablet_manager.cc:1434] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:07:37.193902 25927 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:37.199717 25929 tablet_bootstrap.cc:492] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.200137 25927 tablet_bootstrap.cc:654] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.200307 25929 ts_tablet_manager.cc:1403] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:07:37.203673 25929 raft_consensus.cc:359] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.204285 25929 raft_consensus.cc:385] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.204532 25929 raft_consensus.cc:740] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.205130 25927 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:37.205681 25927 ts_tablet_manager.cc:1403] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.012s	user 0.003s	sys 0.007s
I20260709 13:07:37.205267 25929 consensus_queue.cc:260] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.207499 25929 ts_tablet_manager.cc:1434] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:37.208339 25927 raft_consensus.cc:359] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.208727 25929 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:37.209148 25927 raft_consensus.cc:385] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.209467 25927 raft_consensus.cc:740] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.210145 25927 consensus_queue.cc:260] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.211961 25927 ts_tablet_manager.cc:1434] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:37.213637 25927 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: Bootstrap starting.
I20260709 13:07:37.220700 25929 tablet_bootstrap.cc:654] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.222298 25927 tablet_bootstrap.cc:654] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.235056 25929 tablet_bootstrap.cc:492] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.235531 25929 ts_tablet_manager.cc:1403] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.014s
I20260709 13:07:37.236114 25927 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: No bootstrap required, opened a new log
I20260709 13:07:37.236613 25927 ts_tablet_manager.cc:1403] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:07:37.237846 25929 raft_consensus.cc:359] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.238452 25929 raft_consensus.cc:385] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.238670 25929 raft_consensus.cc:740] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.238729 25927 raft_consensus.cc:359] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.239435 25927 raft_consensus.cc:385] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.239236 25929 consensus_queue.cc:260] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.239661 25927 raft_consensus.cc:740] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.240423 25927 consensus_queue.cc:260] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.241060 25929 ts_tablet_manager.cc:1434] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.005s
I20260709 13:07:37.242050 25929 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: Bootstrap starting.
I20260709 13:07:37.242429 25927 ts_tablet_manager.cc:1434] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:37.246740 25948 raft_consensus.cc:493] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.247287 25948 raft_consensus.cc:515] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.248237 25929 tablet_bootstrap.cc:654] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.249434 25948 leader_election.cc:290] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.251117 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:37.251792 25635 raft_consensus.cc:2468] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:37.252460 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.253160 25784 raft_consensus.cc:2468] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:37.253182 25674 leader_election.cc:304] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.254210 25948 raft_consensus.cc:2804] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.254576 25948 raft_consensus.cc:493] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.254844 25929 tablet_bootstrap.cc:492] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: No bootstrap required, opened a new log
I20260709 13:07:37.254912 25948 raft_consensus.cc:3060] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.255491 25929 ts_tablet_manager.cc:1403] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:07:37.258450 25929 raft_consensus.cc:359] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.259284 25929 raft_consensus.cc:385] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.260385 25929 raft_consensus.cc:740] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.261070 25929 consensus_queue.cc:260] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.262883 25929 ts_tablet_manager.cc:1434] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:37.262634 25948 raft_consensus.cc:515] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.263530 25840 raft_consensus.cc:493] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.264041 25840 raft_consensus.cc:515] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.266193 25840 leader_election.cc:290] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:37.266708 25948 leader_election.cc:290] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.267270 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.267828 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101e50fde3eb4118a8cd3c020c7933e1" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:37.268141 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:37.268452 25635 raft_consensus.cc:3060] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.267885 25784 raft_consensus.cc:3060] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.268972 25710 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:37.268173 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:37.270076 25636 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 0.
I20260709 13:07:37.270452 25750 leader_election.cc:304] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.272141 25840 raft_consensus.cc:2804] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.272541 25840 raft_consensus.cc:493] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.272886 25840 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.276052 25635 raft_consensus.cc:2468] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.277232 25674 leader_election.cc:304] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.278978 25947 raft_consensus.cc:2804] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.279469 25947 raft_consensus.cc:697] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:37.280305 25947 consensus_queue.cc:237] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.280611 25840 raft_consensus.cc:515] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.282526 25840 leader_election.cc:290] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:37.282676 25784 raft_consensus.cc:2468] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.283661 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.284277 25710 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.285091 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:37.285840 25636 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.295576 25710 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:37.295583 25530 catalog_manager.cc:5697] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.297271 25750 leader_election.cc:304] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.298139 25840 raft_consensus.cc:2804] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.299085 25840 raft_consensus.cc:697] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:37.299942 25840 consensus_queue.cc:237] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.300362 25636 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 1.
I20260709 13:07:37.309641 25530 catalog_manager.cc:5697] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 1 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.385342 25947 consensus_queue.cc:1048] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.398912 25948 consensus_queue.cc:1048] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.504920 25956 consensus_queue.cc:1048] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:37.505841 25582 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:37.516381 25956 consensus_queue.cc:1048] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.528090 25948 raft_consensus.cc:493] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.528681 25948 raft_consensus.cc:515] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.531076 25948 leader_election.cc:290] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.532399 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a" is_pre_election: true
I20260709 13:07:37.532682 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.533438 25785 raft_consensus.cc:2468] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:37.533075 25636 raft_consensus.cc:2468] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 0.
I20260709 13:07:37.534554 25677 leader_election.cc:304] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.535763 25948 raft_consensus.cc:2804] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.536183 25948 raft_consensus.cc:493] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.536552 25948 raft_consensus.cc:3060] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.542706 25948 raft_consensus.cc:515] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.544579 25948 leader_election.cc:290] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.545465 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:37.545691 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c252c629b6d480abbec264834467a23" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.546064 25636 raft_consensus.cc:3060] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.546334 25785 raft_consensus.cc:3060] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.551084 25636 raft_consensus.cc:2468] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.551790 25785 raft_consensus.cc:2468] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 1.
I20260709 13:07:37.552289 25674 leader_election.cc:304] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.553244 25948 raft_consensus.cc:2804] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.553678 25948 raft_consensus.cc:697] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:37.554307 25948 consensus_queue.cc:237] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.562386 25530 catalog_manager.cc:5697] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 1 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.570403 25854 consensus_queue.cc:1048] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.582284 25956 consensus_queue.cc:1048] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.587122 25956 raft_consensus.cc:493] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.587761 25956 raft_consensus.cc:515] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.592180 25956 leader_election.cc:290] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.592088 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:37.593149 25709 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.593479 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.594220 25785 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.594565 25601 leader_election.cc:304] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.595499 25956 raft_consensus.cc:2804] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.595830 25956 raft_consensus.cc:493] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.596158 25956 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.600788 25948 consensus_queue.cc:1048] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.603127 25956 raft_consensus.cc:515] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.606081 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.606710 25709 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.607115 25956 leader_election.cc:290] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.610141 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.610924 25784 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.612257 25948 consensus_queue.cc:1048] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.615860 25709 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.617381 25601 leader_election.cc:304] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.618228 25956 raft_consensus.cc:2804] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.618856 25956 raft_consensus.cc:697] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:37.621628 25956 consensus_queue.cc:237] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.631810 25529 catalog_manager.cc:5697] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.619500 25784 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.642359 25956 raft_consensus.cc:493] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.642907 25956 raft_consensus.cc:515] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.644678 25956 leader_election.cc:290] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.645398 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" is_pre_election: true
I20260709 13:07:37.645898 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" is_pre_election: true
I20260709 13:07:37.646092 25709 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.646564 25785 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 0.
I20260709 13:07:37.647351 25601 leader_election.cc:304] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:37.648217 25956 raft_consensus.cc:2804] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.648598 25956 raft_consensus.cc:493] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.649011 25956 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.655570 25956 raft_consensus.cc:515] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.657240 25956 leader_election.cc:290] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.657794 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.658154 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.658396 25709 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.658711 25785 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.663358 25785 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.664147 25709 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 1.
I20260709 13:07:37.664296 25602 leader_election.cc:304] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.665201 25956 raft_consensus.cc:2804] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.665628 25956 raft_consensus.cc:697] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:37.666414 25956 consensus_queue.cc:237] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.674376 25530 catalog_manager.cc:5697] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 1 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.690299 25948 consensus_queue.cc:1048] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.700358 25947 consensus_queue.cc:1048] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.737287 25840 consensus_queue.cc:1048] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.738097 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
mode: GRACEFUL
new_leader_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:07:37.738758 25709 raft_consensus.cc:606] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Received request to transfer leadership to TS 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:37.744514 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
mode: GRACEFUL
new_leader_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:07:37.751619 25709 raft_consensus.cc:606] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Received request to transfer leadership to TS 466dd86270c5461ca8e12b502390e16a
I20260709 13:07:37.757540 25949 consensus_queue.cc:1048] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.770936 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b274557ca07475583c64414a80db13e"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
mode: GRACEFUL
new_leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:54550
I20260709 13:07:37.771498 25636 raft_consensus.cc:606] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Received request to transfer leadership to TS 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:37.776978 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
mode: GRACEFUL
new_leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:44754
I20260709 13:07:37.777637 25785 raft_consensus.cc:606] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Received request to transfer leadership to TS 79c6af590cca4da7b26cbcca34e2cdbe
I20260709 13:07:37.783618 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
mode: GRACEFUL
new_leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:54550
I20260709 13:07:37.784111 25636 raft_consensus.cc:606] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Received request to transfer leadership to TS fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:37.850132 25863 raft_consensus.cc:993] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: : Instructing follower 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:37.850709 25947 raft_consensus.cc:1081] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Signalling peer 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:37.852288 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:38032
I20260709 13:07:37.852802 25636 raft_consensus.cc:493] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.853175 25636 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.857833 25636 raft_consensus.cc:515] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.859325 25636 leader_election.cc:290] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:37.860356 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:37.860371 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8443cc0e7ac464187d88297b95ac54f" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:37.861220 25710 raft_consensus.cc:3055] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.861271 25785 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.861567 25710 raft_consensus.cc:740] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:37.862432 25710 consensus_queue.cc:260] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.863401 25710 raft_consensus.cc:3060] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.866156 25785 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:37.867429 25602 leader_election.cc:304] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:37.868137 25956 raft_consensus.cc:2804] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.868621 25956 raft_consensus.cc:697] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:37.869011 25710 raft_consensus.cc:2468] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:37.869414 25956 consensus_queue.cc:237] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:37.876516 25529 catalog_manager.cc:5697] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 1 to 2, leader changed from 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194) to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 2 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.011636 25863 raft_consensus.cc:993] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: : Instructing follower 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:38.012166 25947 raft_consensus.cc:1081] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Signalling peer 466dd86270c5461ca8e12b502390e16a to start an election
I20260709 13:07:38.013909 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
 from {username='slave'} at 127.0.0.1:38032
I20260709 13:07:38.014374 25636 raft_consensus.cc:493] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:38.014652 25636 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.019366 25636 raft_consensus.cc:515] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.020979 25636 leader_election.cc:290] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Requested vote from peers 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:38.021940 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:38.022189 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16701afc3eee4f9abc7d317abd0c5e3a" candidate_uuid: "466dd86270c5461ca8e12b502390e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:38.022738 25784 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.022738 25710 raft_consensus.cc:3055] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:38.023327 25710 raft_consensus.cc:740] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:38.024183 25710 consensus_queue.cc:260] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.025473 25710 raft_consensus.cc:3060] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.030757 25784 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:38.032075 25602 leader_election.cc:304] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:38.032229 25710 raft_consensus.cc:2468] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466dd86270c5461ca8e12b502390e16a in term 2.
I20260709 13:07:38.032998 25956 raft_consensus.cc:2804] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:38.033555 25956 raft_consensus.cc:697] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Becoming Leader. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:38.034369 25956 consensus_queue.cc:237] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.041194 25529 catalog_manager.cc:5697] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a reported cstate change: term changed from 1 to 2, leader changed from 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194) to 466dd86270c5461ca8e12b502390e16a (127.13.163.193). New cstate: current_term: 2 leader_uuid: "466dd86270c5461ca8e12b502390e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.044598 25947 consensus_queue.cc:1048] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.056532 25948 consensus_queue.cc:1048] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.092423 25956 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:38.100726 25997 raft_consensus.cc:993] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: : Instructing follower 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:38.101330 25955 raft_consensus.cc:1081] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Signalling peer 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:38.103199 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b274557ca07475583c64414a80db13e"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:35824
I20260709 13:07:38.104574 25709 raft_consensus.cc:493] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:38.104998 25709 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.108183 25997 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:38.112205 25709 raft_consensus.cc:515] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.114804 25709 leader_election.cc:290] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:38.116197 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:38.116762 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:38.117455 25635 raft_consensus.cc:3055] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:38.117774 25635 raft_consensus.cc:740] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:38.118680 25635 consensus_queue.cc:260] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.119822 25635 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.126005 25635 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:38.127300 25674 leader_election.cc:304] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:38.128132 25948 raft_consensus.cc:2804] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:38.128638 25948 raft_consensus.cc:697] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:38.129419 25948 consensus_queue.cc:237] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.136262 25530 catalog_manager.cc:5697] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 1 to 2, leader changed from 466dd86270c5461ca8e12b502390e16a (127.13.163.193) to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.153688 25997 consensus_queue.cc:1048] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.161182 25955 raft_consensus.cc:993] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: : Instructing follower fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:38.161725 25955 raft_consensus.cc:1081] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Signalling peer fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:38.168452 25784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:52168
I20260709 13:07:38.169341 25996 consensus_queue.cc:1048] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:38.171823 25784 raft_consensus.cc:493] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:38.172379 25784 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.180442 25784 raft_consensus.cc:515] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.182642 25784 leader_election.cc:290] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:38.183408 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:38.183755 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9080c81947294f0b82051bbe4b2e1670" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:38.184157 25635 raft_consensus.cc:3055] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:38.184391 25709 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.184576 25635 raft_consensus.cc:740] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:38.185604 25635 consensus_queue.cc:260] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.189507 25635 raft_consensus.cc:3060] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.194067 25709 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:38.195259 25750 leader_election.cc:304] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:38.196048 25991 raft_consensus.cc:2804] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:38.196455 25991 raft_consensus.cc:697] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:38.196504 25635 raft_consensus.cc:2468] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:38.197345 25991 consensus_queue.cc:237] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.204516 25529 catalog_manager.cc:5697] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 1 to 2, leader changed from 466dd86270c5461ca8e12b502390e16a (127.13.163.193) to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 2 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.263866 25991 raft_consensus.cc:993] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: : Instructing follower 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:38.264443 25992 raft_consensus.cc:1081] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Signalling peer 79c6af590cca4da7b26cbcca34e2cdbe to start an election
I20260709 13:07:38.266047 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
 from {username='slave'} at 127.0.0.1:35830
I20260709 13:07:38.266628 25709 raft_consensus.cc:493] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:38.267046 25709 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.272162 25709 raft_consensus.cc:515] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.273773 25709 leader_election.cc:290] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195:33817)
I20260709 13:07:38.274653 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:38.274878 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c381f41065ab4d7fbfb26b80cc3d5287" candidate_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
I20260709 13:07:38.275336 25635 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.275511 25784 raft_consensus.cc:3055] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:38.275883 25784 raft_consensus.cc:740] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:38.276582 25784 consensus_queue.cc:260] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.277693 25784 raft_consensus.cc:3060] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.281234 25635 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:38.282289 25674 leader_election.cc:304] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, 79c6af590cca4da7b26cbcca34e2cdbe; no voters: 
I20260709 13:07:38.282982 25948 raft_consensus.cc:2804] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:38.283075 25784 raft_consensus.cc:2468] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c6af590cca4da7b26cbcca34e2cdbe in term 2.
I20260709 13:07:38.283408 25948 raft_consensus.cc:697] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Becoming Leader. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:38.284296 25948 consensus_queue.cc:237] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:38.291669 25530 catalog_manager.cc:5697] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe reported cstate change: term changed from 1 to 2, leader changed from fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195) to 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194). New cstate: current_term: 2 leader_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.420069 25784 raft_consensus.cc:1275] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.420766 25709 raft_consensus.cc:1275] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.421489 25996 consensus_queue.cc:1048] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.422390 25956 consensus_queue.cc:1048] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.429673 25709 raft_consensus.cc:1275] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.431731 25955 consensus_queue.cc:1048] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.444099 25781 raft_consensus.cc:1275] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 466dd86270c5461ca8e12b502390e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.448140 25956 consensus_queue.cc:1048] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.512137 25635 raft_consensus.cc:1275] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.514271 25948 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.522814 25784 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.533067 25784 raft_consensus.cc:1275] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.534471 25947 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.660321 25635 raft_consensus.cc:1275] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.661899 25992 consensus_queue.cc:1048] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.670032 25710 raft_consensus.cc:1275] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.671638 25992 consensus_queue.cc:1048] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.737051 25781 raft_consensus.cc:1275] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.738644 25947 consensus_queue.cc:1048] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.746655 25635 raft_consensus.cc:1275] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer 79c6af590cca4da7b26cbcca34e2cdbe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.748481 26006 consensus_queue.cc:1048] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:40.787853 25533 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51066:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:07:40.808708 25533 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51066:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:07:40.902808 13967 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:07:40.949106 25602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:33817, user_credentials={real_user=slave}} blocked reactor thread for 71550us
I20260709 13:07:41.014493 26038 tablet.cc:1724] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.022753 26040 tablet.cc:1724] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.070611 26058 tablet.cc:1724] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.081351 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:33817, user_credentials={real_user=slave}} blocked reactor thread for 55231.7us
W20260709 13:07:41.105754 25662 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:16701afc3eee4f9abc7d317abd0c5e3a)
I20260709 13:07:41.158502 26061 tablet.cc:1724] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.179142 25662 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:16701afc3eee4f9abc7d317abd0c5e3a)
W20260709 13:07:41.180097 25662 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:16701afc3eee4f9abc7d317abd0c5e3a)
I20260709 13:07:41.208999 26064 tablet.cc:1724] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.170418 26054 tablet.cc:1724] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.267392 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 74272.7us
I20260709 13:07:41.238458 26071 tablet.cc:1724] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.261415 26070 tablet.cc:1724] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.304437 26038 tablet.cc:1724] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.310847 26079 tablet.cc:1724] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.290496 26065 tablet.cc:1724] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.349866 26058 tablet.cc:1724] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.352491 26072 log.cc:927] Time spent T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Append to log took a long time: real 0.089s	user 0.002s	sys 0.001s
W20260709 13:07:41.353317 26074 log.cc:927] Time spent T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:07:41.351822 26061 tablet.cc:1724] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.367012 26073 log.cc:927] Time spent T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:07:41.368599 25710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b274557ca07475583c64414a80db13e"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
mode: GRACEFUL
new_leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:07:41.369695 25710 raft_consensus.cc:606] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Received request to transfer leadership to TS fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:41.409989 26040 tablet.cc:1724] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.414948 26058 tablet.cc:1724] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.424459 26065 tablet.cc:1724] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.428601 26040 tablet.cc:1724] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.435389 26078 tablet.cc:1724] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.440243 26040 tablet.cc:1724] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.464537 26081 tablet.cc:1724] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.465104 26054 tablet.cc:1724] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.468561 26070 tablet.cc:1724] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.474260 26058 tablet.cc:1724] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.487459 26065 tablet.cc:1724] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:41.491741 26083 log.cc:927] Time spent T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:07:41.492185 25633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
mode: GRACEFUL
new_leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:54566
I20260709 13:07:41.492993 25633 raft_consensus.cc:606] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Received request to transfer leadership to TS fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:41.493713 26079 tablet.cc:1724] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.496104 13967 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:07:41.495704 26054 tablet.cc:1724] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.496924 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:41.514323 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf"
dest_uuid: "466dd86270c5461ca8e12b502390e16a"
mode: GRACEFUL
new_leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:54566
I20260709 13:07:41.515154 25632 raft_consensus.cc:606] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Received request to transfer leadership to TS fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:41.519398 25711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f"
dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
mode: GRACEFUL
new_leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:07:41.520043 25711 raft_consensus.cc:606] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Received request to transfer leadership to TS fae7ec30f9db4a42a639dfc6a1757b2f
I20260709 13:07:41.512398 26038 tablet.cc:1724] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:41.521865 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:41.527750 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:41.545241 25530 catalog_manager.cc:6242] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: soft_deleted_table [id=b0236cb8c7c449ad9afe96f4f307053d] alter complete (version 1)
I20260709 13:07:41.590408 26030 raft_consensus.cc:993] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: : Instructing follower fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.591114 26042 raft_consensus.cc:1081] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Signalling peer fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.592823 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:52168
I20260709 13:07:41.593466 25779 raft_consensus.cc:493] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:41.593868 25779 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.601015 25779 raft_consensus.cc:515] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.603497 25779 leader_election.cc:290] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:41.604140 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:41.604950 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe634984c1149f78138ccff0f419bcf" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:41.605579 25711 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.604988 25632 raft_consensus.cc:3055] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:41.609458 25632 raft_consensus.cc:740] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:41.610378 25632 consensus_queue.cc:260] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.611474 25711 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.611722 25632 raft_consensus.cc:3060] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.613066 25750 leader_election.cc:304] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:41.614058 26077 raft_consensus.cc:2804] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:41.614598 26077 raft_consensus.cc:697] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:41.615442 26077 consensus_queue.cc:237] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.618719 25632 raft_consensus.cc:2468] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.624797 25529 catalog_manager.cc:5697] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 1 to 2, leader changed from 466dd86270c5461ca8e12b502390e16a (127.13.163.193) to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 2 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.653297 25947 raft_consensus.cc:993] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: : Instructing follower fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.653851 26006 raft_consensus.cc:1081] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Signalling peer fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.655359 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:52180
I20260709 13:07:41.655864 25779 raft_consensus.cc:493] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:41.656116 25779 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.661104 25779 raft_consensus.cc:515] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.662938 25779 leader_election.cc:290] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:41.663890 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:41.664002 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2c18175044c57b773b2f27526fe2f" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:41.664606 25632 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.664640 25711 raft_consensus.cc:3055] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:41.665132 25711 raft_consensus.cc:740] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:41.665803 25711 consensus_queue.cc:260] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.666946 25711 raft_consensus.cc:3060] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.671233 25632 raft_consensus.cc:2468] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.672362 25748 leader_election.cc:304] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:41.673195 26077 raft_consensus.cc:2804] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:41.673434 25711 raft_consensus.cc:2468] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.673724 26077 raft_consensus.cc:697] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:41.674872 26077 consensus_queue.cc:237] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.682760 25529 catalog_manager.cc:5697] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 1 to 2, leader changed from 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194) to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 2 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.730501 26030 raft_consensus.cc:993] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: : Instructing follower fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.731014 26042 raft_consensus.cc:1081] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Signalling peer fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.732489 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:52168
I20260709 13:07:41.733117 25779 raft_consensus.cc:493] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:41.733498 25779 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.740233 25779 raft_consensus.cc:515] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.742261 25779 leader_election.cc:290] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:41.742776 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:41.743345 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4c41ba20d54b4bb7a76a6c68f118e9" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:41.743513 25633 raft_consensus.cc:3055] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:41.744019 25711 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.744033 25633 raft_consensus.cc:740] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 466dd86270c5461ca8e12b502390e16a, State: Running, Role: LEADER
I20260709 13:07:41.745023 25633 consensus_queue.cc:260] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.745990 25633 raft_consensus.cc:3060] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.749503 25711 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.750892 25750 leader_election.cc:304] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6af590cca4da7b26cbcca34e2cdbe, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:41.751623 26077 raft_consensus.cc:2804] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:41.752115 26077 raft_consensus.cc:697] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:41.752507 25633 raft_consensus.cc:2468] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 2.
I20260709 13:07:41.753016 26077 consensus_queue.cc:237] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.759553 25529 catalog_manager.cc:5697] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 1 to 2, leader changed from 466dd86270c5461ca8e12b502390e16a (127.13.163.193) to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 2 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.874835 26048 raft_consensus.cc:993] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: : Instructing follower fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.875329 26006 raft_consensus.cc:1081] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Signalling peer fae7ec30f9db4a42a639dfc6a1757b2f to start an election
I20260709 13:07:41.877108 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b274557ca07475583c64414a80db13e"
dest_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f"
 from {username='slave'} at 127.0.0.1:52180
I20260709 13:07:41.877626 25779 raft_consensus.cc:493] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:41.877902 25779 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:41.883190 25779 raft_consensus.cc:515] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.884945 25779 leader_election.cc:290] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 3 election: Requested vote from peers 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187), 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649)
I20260709 13:07:41.886000 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "79c6af590cca4da7b26cbcca34e2cdbe"
I20260709 13:07:41.885993 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b274557ca07475583c64414a80db13e" candidate_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "466dd86270c5461ca8e12b502390e16a"
I20260709 13:07:41.886845 25711 raft_consensus.cc:3055] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:41.886981 25633 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:41.887215 25711 raft_consensus.cc:740] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 79c6af590cca4da7b26cbcca34e2cdbe, State: Running, Role: LEADER
I20260709 13:07:41.888285 25711 consensus_queue.cc:260] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.889514 25711 raft_consensus.cc:3060] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:41.893483 25633 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 3.
I20260709 13:07:41.894619 25748 leader_election.cc:304] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 466dd86270c5461ca8e12b502390e16a, fae7ec30f9db4a42a639dfc6a1757b2f; no voters: 
I20260709 13:07:41.894912 25711 raft_consensus.cc:2468] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae7ec30f9db4a42a639dfc6a1757b2f in term 3.
I20260709 13:07:41.895370 26077 raft_consensus.cc:2804] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:41.895946 26077 raft_consensus.cc:697] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 3 LEADER]: Becoming Leader. State: Replica: fae7ec30f9db4a42a639dfc6a1757b2f, State: Running, Role: LEADER
I20260709 13:07:41.896764 26077 consensus_queue.cc:237] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } }
I20260709 13:07:41.903837 25529 catalog_manager.cc:5697] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f reported cstate change: term changed from 2 to 3, leader changed from 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194) to fae7ec30f9db4a42a639dfc6a1757b2f (127.13.163.195). New cstate: current_term: 3 leader_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae7ec30f9db4a42a639dfc6a1757b2f" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 33817 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.055425 25711 raft_consensus.cc:1275] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.056366 25710 raft_consensus.cc:1275] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.056980 26077 consensus_queue.cc:1048] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:42.057780 26069 consensus_queue.cc:1048] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:42.082604 25633 raft_consensus.cc:1275] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.083666 25632 raft_consensus.cc:1275] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.085807 26094 consensus_queue.cc:1048] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:42.086313 26069 consensus_queue.cc:1048] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:42.137522 25711 raft_consensus.cc:1275] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:42.139379 26093 consensus_queue.cc:1048] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:42.151629 25632 raft_consensus.cc:1275] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:42.154309 26094 consensus_queue.cc:1048] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:42.310691 25632 raft_consensus.cc:1275] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 3 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:42.312435 26094 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "466dd86270c5461ca8e12b502390e16a" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:42.321326 25711 raft_consensus.cc:1275] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 3 FOLLOWER]: Refusing update from remote peer fae7ec30f9db4a42a639dfc6a1757b2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:42.323392 26077 consensus_queue.cc:1048] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6af590cca4da7b26cbcca34e2cdbe" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:42.591740 13967 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:07:42.599462 13967 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:07:42.600198 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:42.607400 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:07:42.607942 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:43.639469 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
W20260709 13:07:43.660297 25748 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260709 13:07:43.665277 25748 consensus_peers.cc:597] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.670146 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:43.671051 13967 tablet_replica.cc:333] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.671778 13967 raft_consensus.cc:2243] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.672271 13967 raft_consensus.cc:2272] T c381f41065ab4d7fbfb26b80cc3d5287 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.674376 13967 tablet_replica.cc:333] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.674957 13967 raft_consensus.cc:2243] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.675434 13967 raft_consensus.cc:2272] T 9080c81947294f0b82051bbe4b2e1670 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.677424 13967 tablet_replica.cc:333] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.677986 13967 raft_consensus.cc:2243] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.678443 13967 raft_consensus.cc:2272] T 1b274557ca07475583c64414a80db13e P 466dd86270c5461ca8e12b502390e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.680480 13967 tablet_replica.cc:333] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.681239 13967 raft_consensus.cc:2243] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.681747 13967 raft_consensus.cc:2272] T 3c252c629b6d480abbec264834467a23 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.683490 13967 tablet_replica.cc:333] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.684049 13967 raft_consensus.cc:2243] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.684468 13967 raft_consensus.cc:2272] T 101e50fde3eb4118a8cd3c020c7933e1 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.686265 13967 tablet_replica.cc:333] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.686802 13967 raft_consensus.cc:2243] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.687204 13967 raft_consensus.cc:2272] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.689311 13967 tablet_replica.cc:333] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.689836 13967 raft_consensus.cc:2243] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.690846 13967 raft_consensus.cc:2272] T c8443cc0e7ac464187d88297b95ac54f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.692593 13967 tablet_replica.cc:333] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.693202 13967 raft_consensus.cc:2243] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.693986 13967 raft_consensus.cc:2272] T ab53fc1ca0f149dca173e9867389ae94 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.695871 13967 tablet_replica.cc:333] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.696461 13967 raft_consensus.cc:2243] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.697355 13967 raft_consensus.cc:2272] T 902c2b59aab64796be18c4dc3adaca06 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.699806 13967 tablet_replica.cc:333] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.700690 13967 raft_consensus.cc:2243] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.701218 13967 raft_consensus.cc:2272] T 175d87f9e1d04e73bf1464b3070e7370 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.703593 13967 tablet_replica.cc:333] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.704187 13967 raft_consensus.cc:2243] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.704635 13967 raft_consensus.cc:2272] T fca2c18175044c57b773b2f27526fe2f P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.706866 13967 tablet_replica.cc:333] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.707391 13967 raft_consensus.cc:2243] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.708168 13967 raft_consensus.cc:2272] T 16701afc3eee4f9abc7d317abd0c5e3a P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.710126 13967 tablet_replica.cc:333] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.710696 13967 raft_consensus.cc:2243] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.711166 13967 raft_consensus.cc:2272] T 0fe634984c1149f78138ccff0f419bcf P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.713944 13967 tablet_replica.cc:333] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.714618 13967 raft_consensus.cc:2243] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.715600 13967 raft_consensus.cc:2272] T ddbb0f118713403e92a23cf7f1d71eda P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:43.716738 25674 consensus_peers.cc:597] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.718168 13967 tablet_replica.cc:333] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.718832 13967 raft_consensus.cc:2243] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.719310 13967 raft_consensus.cc:2272] T b3fb62cb171046f1a45df62a2c4efcff P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.721781 13967 tablet_replica.cc:333] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.722499 13967 raft_consensus.cc:2243] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.723479 13967 raft_consensus.cc:2272] T daae2603a8ad48b4ad155c7971a07739 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.726502 13967 tablet_replica.cc:333] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.727226 13967 raft_consensus.cc:2243] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.727774 13967 raft_consensus.cc:2272] T 5f1414dc93b84fa49258bd56a9334f65 P 466dd86270c5461ca8e12b502390e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.730166 13967 tablet_replica.cc:333] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a: stopping tablet replica
I20260709 13:07:43.730634 13967 raft_consensus.cc:2243] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.730998 13967 raft_consensus.cc:2272] T 9bdbf339b8f941baaffa2d12334932f0 P 466dd86270c5461ca8e12b502390e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.771534 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
W20260709 13:07:43.784617 25674 consensus_peers.cc:597] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:43187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:43.806285 25674 consensus_peers.cc:597] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:43187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.808341 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
W20260709 13:07:43.830454 25750 consensus_peers.cc:597] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f -> Peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Couldn't send request to peer 79c6af590cca4da7b26cbcca34e2cdbe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.848265 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:43.849150 13967 tablet_replica.cc:333] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.849788 13967 raft_consensus.cc:2243] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.850646 13967 raft_consensus.cc:2272] T c381f41065ab4d7fbfb26b80cc3d5287 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.852614 13967 tablet_replica.cc:333] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.853246 13967 raft_consensus.cc:2243] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.853713 13967 raft_consensus.cc:2272] T 9080c81947294f0b82051bbe4b2e1670 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.855458 13967 tablet_replica.cc:333] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.856055 13967 raft_consensus.cc:2243] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.856498 13967 raft_consensus.cc:2272] T 1b274557ca07475583c64414a80db13e P 79c6af590cca4da7b26cbcca34e2cdbe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.858384 13967 tablet_replica.cc:333] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.858948 13967 raft_consensus.cc:2243] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.859737 13967 raft_consensus.cc:2272] T 3c252c629b6d480abbec264834467a23 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.861655 13967 tablet_replica.cc:333] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.862202 13967 raft_consensus.cc:2243] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.862969 13967 raft_consensus.cc:2272] T 101e50fde3eb4118a8cd3c020c7933e1 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.864797 13967 tablet_replica.cc:333] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.865499 13967 raft_consensus.cc:2243] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.866056 13967 raft_consensus.cc:2272] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.868359 13967 tablet_replica.cc:333] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
W20260709 13:07:43.868682 25750 consensus_peers.cc:597] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f -> Peer 79c6af590cca4da7b26cbcca34e2cdbe (127.13.163.194:33649): Couldn't send request to peer 79c6af590cca4da7b26cbcca34e2cdbe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.869016 13967 raft_consensus.cc:2243] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.869626 13967 raft_consensus.cc:2272] T ab53fc1ca0f149dca173e9867389ae94 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.871572 13967 tablet_replica.cc:333] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.872184 13967 raft_consensus.cc:2243] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.873049 13967 raft_consensus.cc:2272] T 175d87f9e1d04e73bf1464b3070e7370 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.875121 13967 tablet_replica.cc:333] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.875730 13967 raft_consensus.cc:2243] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.876195 13967 raft_consensus.cc:2272] T 902c2b59aab64796be18c4dc3adaca06 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.878281 13967 tablet_replica.cc:333] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.878841 13967 raft_consensus.cc:2243] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.879721 13967 raft_consensus.cc:2272] T b3fb62cb171046f1a45df62a2c4efcff P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.881866 13967 tablet_replica.cc:333] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.882429 13967 raft_consensus.cc:2243] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.882897 13967 raft_consensus.cc:2272] T daae2603a8ad48b4ad155c7971a07739 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.885231 13967 tablet_replica.cc:333] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.885766 13967 raft_consensus.cc:2243] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.886209 13967 raft_consensus.cc:2272] T 5f1414dc93b84fa49258bd56a9334f65 P 79c6af590cca4da7b26cbcca34e2cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.888177 13967 tablet_replica.cc:333] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.888700 13967 raft_consensus.cc:2243] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:43.889492 13967 raft_consensus.cc:2272] T 9bdbf339b8f941baaffa2d12334932f0 P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.891633 13967 tablet_replica.cc:333] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.892174 13967 raft_consensus.cc:2243] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.892591 13967 raft_consensus.cc:2272] T fca2c18175044c57b773b2f27526fe2f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.894472 13967 tablet_replica.cc:333] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.894996 13967 raft_consensus.cc:2243] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.895416 13967 raft_consensus.cc:2272] T c8443cc0e7ac464187d88297b95ac54f P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.897215 13967 tablet_replica.cc:333] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.897719 13967 raft_consensus.cc:2243] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.898187 13967 raft_consensus.cc:2272] T ddbb0f118713403e92a23cf7f1d71eda P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.900465 13967 tablet_replica.cc:333] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.901062 13967 raft_consensus.cc:2243] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.901697 13967 raft_consensus.cc:2272] T 0fe634984c1149f78138ccff0f419bcf P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:43.903576 25748 consensus_peers.cc:597] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:43187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.903928 13967 tablet_replica.cc:333] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe: stopping tablet replica
I20260709 13:07:43.904553 13967 raft_consensus.cc:2243] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:43.905054 13967 raft_consensus.cc:2272] T 16701afc3eee4f9abc7d317abd0c5e3a P 79c6af590cca4da7b26cbcca34e2cdbe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:43.947072 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
I20260709 13:07:43.978695 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
W20260709 13:07:43.985194 25748 consensus_peers.cc:597] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f -> Peer 466dd86270c5461ca8e12b502390e16a (127.13.163.193:43187): Couldn't send request to peer 466dd86270c5461ca8e12b502390e16a. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:43187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:44.008194 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:44.009109 13967 tablet_replica.cc:333] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.009861 13967 raft_consensus.cc:2243] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.010727 13967 raft_consensus.cc:2272] T 9080c81947294f0b82051bbe4b2e1670 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.012724 13967 tablet_replica.cc:333] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.013350 13967 raft_consensus.cc:2243] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.013901 13967 raft_consensus.cc:2272] T c381f41065ab4d7fbfb26b80cc3d5287 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.015769 13967 tablet_replica.cc:333] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.016320 13967 raft_consensus.cc:2243] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.017198 13967 raft_consensus.cc:2272] T 1b274557ca07475583c64414a80db13e P fae7ec30f9db4a42a639dfc6a1757b2f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.018987 13967 tablet_replica.cc:333] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.019531 13967 raft_consensus.cc:2243] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.019953 13967 raft_consensus.cc:2272] T 3c252c629b6d480abbec264834467a23 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.021780 13967 tablet_replica.cc:333] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.022306 13967 raft_consensus.cc:2243] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.022735 13967 raft_consensus.cc:2272] T 101e50fde3eb4118a8cd3c020c7933e1 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.024499 13967 tablet_replica.cc:333] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.025092 13967 raft_consensus.cc:2243] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.025545 13967 raft_consensus.cc:2272] T c8443cc0e7ac464187d88297b95ac54f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.027436 13967 tablet_replica.cc:333] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.028007 13967 raft_consensus.cc:2243] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.028782 13967 raft_consensus.cc:2272] T 7d4c41ba20d54b4bb7a76a6c68f118e9 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.030820 13967 tablet_replica.cc:333] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.031390 13967 raft_consensus.cc:2243] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.031817 13967 raft_consensus.cc:2272] T ab53fc1ca0f149dca173e9867389ae94 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.033732 13967 tablet_replica.cc:333] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.034258 13967 raft_consensus.cc:2243] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.034685 13967 raft_consensus.cc:2272] T 16701afc3eee4f9abc7d317abd0c5e3a P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.036509 13967 tablet_replica.cc:333] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.037101 13967 raft_consensus.cc:2243] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.037861 13967 raft_consensus.cc:2272] T 0fe634984c1149f78138ccff0f419bcf P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.039937 13967 tablet_replica.cc:333] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.040453 13967 raft_consensus.cc:2243] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.040942 13967 raft_consensus.cc:2272] T ddbb0f118713403e92a23cf7f1d71eda P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.042783 13967 tablet_replica.cc:333] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.043285 13967 raft_consensus.cc:2243] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.043715 13967 raft_consensus.cc:2272] T 902c2b59aab64796be18c4dc3adaca06 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.045644 13967 tablet_replica.cc:333] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.046150 13967 raft_consensus.cc:2243] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.046571 13967 raft_consensus.cc:2272] T 175d87f9e1d04e73bf1464b3070e7370 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.048544 13967 tablet_replica.cc:333] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.049125 13967 raft_consensus.cc:2243] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.049890 13967 raft_consensus.cc:2272] T fca2c18175044c57b773b2f27526fe2f P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.051995 13967 tablet_replica.cc:333] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.052522 13967 raft_consensus.cc:2243] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.053261 13967 raft_consensus.cc:2272] T 5f1414dc93b84fa49258bd56a9334f65 P fae7ec30f9db4a42a639dfc6a1757b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.055121 13967 tablet_replica.cc:333] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.055655 13967 raft_consensus.cc:2243] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.056084 13967 raft_consensus.cc:2272] T daae2603a8ad48b4ad155c7971a07739 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.058175 13967 tablet_replica.cc:333] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.058691 13967 raft_consensus.cc:2243] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.059115 13967 raft_consensus.cc:2272] T b3fb62cb171046f1a45df62a2c4efcff P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.061167 13967 tablet_replica.cc:333] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f: stopping tablet replica
I20260709 13:07:44.061677 13967 raft_consensus.cc:2243] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:44.062137 13967 raft_consensus.cc:2272] T 9bdbf339b8f941baaffa2d12334932f0 P fae7ec30f9db4a42a639dfc6a1757b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.101835 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:07:44.133347 13967 master.cc:562] Master@127.13.163.254:37557 shutting down...
I20260709 13:07:44.155495 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:44.156072 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:44.156410 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P a5e911fd20f24bcdbe43b8cb02ba0948: stopping tablet replica
I20260709 13:07:44.176115 13967 master.cc:584] Master@127.13.163.254:37557 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12941 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:07:44.229274 13967 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.13.163.254:44849
I20260709 13:07:44.230468 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:44.236413 26105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.236512 26106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.237360 26108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:44.238261 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:44.239287 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:44.239593 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:44.239743 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602464239727 us; error 0 us; skew 500 ppm
I20260709 13:07:44.240342 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:44.243048 13967 webserver.cc:533] Webserver started at http://127.13.163.254:39603/ using document root <none> and password file <none>
I20260709 13:07:44.243619 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:44.243791 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:44.244122 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:44.245708 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/master-0-root/instance:
uuid: "602f3b8e5a9842c1bf5ff513d83246e3"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.250607 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:07:44.254500 26113 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.255487 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:44.255759 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/master-0-root
uuid: "602f3b8e5a9842c1bf5ff513d83246e3"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.256004 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:44.275137 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:44.276332 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:44.315842 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.254:44849
I20260709 13:07:44.316068 26164 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.254:44849 every 8 connection(s)
I20260709 13:07:44.320367 26165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.332479 26165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: Bootstrap starting.
I20260709 13:07:44.337806 26165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.342780 26165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: No bootstrap required, opened a new log
I20260709 13:07:44.345378 26165 raft_consensus.cc:359] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER }
I20260709 13:07:44.345863 26165 raft_consensus.cc:385] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.346104 26165 raft_consensus.cc:740] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602f3b8e5a9842c1bf5ff513d83246e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.346976 26165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER }
I20260709 13:07:44.347549 26165 raft_consensus.cc:399] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:44.347777 26165 raft_consensus.cc:493] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:44.348122 26165 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.353875 26165 raft_consensus.cc:515] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER }
I20260709 13:07:44.354537 26165 leader_election.cc:304] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 602f3b8e5a9842c1bf5ff513d83246e3; no voters: 
I20260709 13:07:44.355787 26165 leader_election.cc:290] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:44.356240 26168 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.357801 26168 raft_consensus.cc:697] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 1 LEADER]: Becoming Leader. State: Replica: 602f3b8e5a9842c1bf5ff513d83246e3, State: Running, Role: LEADER
I20260709 13:07:44.358527 26168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER }
I20260709 13:07:44.359230 26165 sys_catalog.cc:565] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:44.362699 26169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER } }
I20260709 13:07:44.363485 26169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:44.362733 26170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602f3b8e5a9842c1bf5ff513d83246e3. Latest consensus state: current_term: 1 leader_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f3b8e5a9842c1bf5ff513d83246e3" member_type: VOTER } }
I20260709 13:07:44.364144 26170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:44.366832 26175 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:44.371779 26175 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:44.376308 13967 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:44.381515 26175 catalog_manager.cc:1383] Generated new cluster ID: dc0fb97e39964ade857a648779dd0c24
I20260709 13:07:44.381824 26175 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:44.399377 26175 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:44.400785 26175 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:44.419049 26175 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: Generated new TSK 0
I20260709 13:07:44.419780 26175 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:44.446666 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:44.452696 26186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.453667 26187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:44.455684 13967 server_base.cc:1061] running on GCE node
W20260709 13:07:44.455641 26189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:44.456810 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:44.457060 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:44.457260 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602464457225 us; error 0 us; skew 500 ppm
I20260709 13:07:44.457958 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:44.460295 13967 webserver.cc:533] Webserver started at http://127.13.163.193:45659/ using document root <none> and password file <none>
I20260709 13:07:44.460803 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:44.461129 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:44.461359 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:44.462378 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-0-root/instance:
uuid: "e600c189c03f439abeed018164fc6d39"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.466913 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:07:44.470623 26194 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.471524 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:44.471849 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-0-root
uuid: "e600c189c03f439abeed018164fc6d39"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.472137 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:44.516788 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:44.518739 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:44.520514 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:44.523391 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:44.523640 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.523963 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:44.524109 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.572356 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.193:45401
I20260709 13:07:44.572556 26257 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.193:45401 every 8 connection(s)
I20260709 13:07:44.579378 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:44.588723 26262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.588871 26263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.593184 26265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:44.594163 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:44.595811 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:44.596074 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:44.596331 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602464596308 us; error 0 us; skew 500 ppm
I20260709 13:07:44.597074 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:44.600754 13967 webserver.cc:533] Webserver started at http://127.13.163.194:45657/ using document root <none> and password file <none>
I20260709 13:07:44.601671 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:44.601995 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:44.602510 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:44.602643 26258 heartbeater.cc:344] Connected to a master server at 127.13.163.254:44849
I20260709 13:07:44.603154 26258 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:44.604084 26258 heartbeater.cc:507] Master 127.13.163.254:44849 requested a full tablet report, sending...
I20260709 13:07:44.604480 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-1-root/instance:
uuid: "e72cb39b26f7438abc76d6f96e894597"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.607146 26130 ts_manager.cc:194] Registered new tserver with Master: e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401)
I20260709 13:07:44.610746 26130 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48160
I20260709 13:07:44.612566 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:44.617092 26270 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.618009 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:07:44.618358 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-1-root
uuid: "e72cb39b26f7438abc76d6f96e894597"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.618746 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:44.631448 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:44.632769 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:44.634410 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:44.636770 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:44.637085 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.637354 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:44.637539 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.681187 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.194:40625
I20260709 13:07:44.681304 26332 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.194:40625 every 8 connection(s)
I20260709 13:07:44.686512 13967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:44.695399 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.697715 26337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:44.699443 26339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:44.701155 26333 heartbeater.cc:344] Connected to a master server at 127.13.163.254:44849
I20260709 13:07:44.701713 26333 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:44.701848 13967 server_base.cc:1061] running on GCE node
I20260709 13:07:44.702790 26333 heartbeater.cc:507] Master 127.13.163.254:44849 requested a full tablet report, sending...
I20260709 13:07:44.702996 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:44.703383 13967 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:44.703601 13967 hybrid_clock.cc:648] HybridClock initialized: now 1783602464703582 us; error 0 us; skew 500 ppm
I20260709 13:07:44.704452 13967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:44.705318 26130 ts_manager.cc:194] Registered new tserver with Master: e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:44.707695 13967 webserver.cc:533] Webserver started at http://127.13.163.195:36849/ using document root <none> and password file <none>
I20260709 13:07:44.707767 26130 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48170
I20260709 13:07:44.708725 13967 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:44.709066 13967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:44.709429 13967 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:44.710992 13967 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-2-root/instance:
uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.716408 13967 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:07:44.720674 26344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.721763 13967 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:44.722113 13967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-2-root
uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
format_stamp: "Formatted at 2026-07-09 13:07:44 on dist-test-slave-w3b1"
I20260709 13:07:44.722422 13967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskznDap5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602092437201-13967-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:44.742399 13967 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:44.743767 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:44.745527 13967 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:44.748199 13967 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:44.748412 13967 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:44.748754 13967 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:44.749017 13967 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:44.797109 13967 rpc_server.cc:307] RPC server started. Bound to: 127.13.163.195:45787
I20260709 13:07:44.797205 26406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.163.195:45787 every 8 connection(s)
I20260709 13:07:44.813537 26407 heartbeater.cc:344] Connected to a master server at 127.13.163.254:44849
I20260709 13:07:44.813985 26407 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:44.814853 26407 heartbeater.cc:507] Master 127.13.163.254:44849 requested a full tablet report, sending...
I20260709 13:07:44.817333 26130 ts_manager.cc:194] Registered new tserver with Master: a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:44.818455 13967 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017444914s
I20260709 13:07:44.819131 26130 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48182
I20260709 13:07:44.848821 26130 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48184:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:44.851958 26130 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:44.932201 26222 tablet_service.cc:1511] Processing CreateTablet for tablet 490b6ffef368438591943bc51cc81736 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:44.933378 26223 tablet_service.cc:1511] Processing CreateTablet for tablet d54584a4053b41c99b53dda055ba490d (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:44.934056 26222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490b6ffef368438591943bc51cc81736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.934939 26223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54584a4053b41c99b53dda055ba490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.937821 26221 tablet_service.cc:1511] Processing CreateTablet for tablet c0b03910d4d549828b82cf0ea4fe938b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:44.939546 26221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b03910d4d549828b82cf0ea4fe938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.944468 26220 tablet_service.cc:1511] Processing CreateTablet for tablet 43c027ecfd3b4d3e889ef85da3520df5 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:44.946200 26220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c027ecfd3b4d3e889ef85da3520df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.955381 26219 tablet_service.cc:1511] Processing CreateTablet for tablet 95a57fd9a10946479d74f32616f5437b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:44.958122 26218 tablet_service.cc:1511] Processing CreateTablet for tablet cae45edb272b44a9aaaa37818f4c2ce7 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:44.959405 26216 tablet_service.cc:1511] Processing CreateTablet for tablet 04c9919a91214947966bdec9eb37aec4 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:44.960625 26217 tablet_service.cc:1511] Processing CreateTablet for tablet 8508c52406f8476ba4da651c8b8b7d37 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:44.962193 26219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95a57fd9a10946479d74f32616f5437b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.965308 26217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8508c52406f8476ba4da651c8b8b7d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.969298 26218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae45edb272b44a9aaaa37818f4c2ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.971271 26298 tablet_service.cc:1511] Processing CreateTablet for tablet d54584a4053b41c99b53dda055ba490d (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:44.972802 26298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54584a4053b41c99b53dda055ba490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.973831 26216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c9919a91214947966bdec9eb37aec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.986778 26297 tablet_service.cc:1511] Processing CreateTablet for tablet 490b6ffef368438591943bc51cc81736 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:44.988391 26297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490b6ffef368438591943bc51cc81736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:44.991580 26372 tablet_service.cc:1511] Processing CreateTablet for tablet d54584a4053b41c99b53dda055ba490d (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:44.993268 26369 tablet_service.cc:1511] Processing CreateTablet for tablet 43c027ecfd3b4d3e889ef85da3520df5 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:44.994076 26371 tablet_service.cc:1511] Processing CreateTablet for tablet 490b6ffef368438591943bc51cc81736 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:44.995800 26368 tablet_service.cc:1511] Processing CreateTablet for tablet 95a57fd9a10946479d74f32616f5437b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:44.996524 26370 tablet_service.cc:1511] Processing CreateTablet for tablet c0b03910d4d549828b82cf0ea4fe938b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:44.998018 26367 tablet_service.cc:1511] Processing CreateTablet for tablet cae45edb272b44a9aaaa37818f4c2ce7 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:45.000133 26366 tablet_service.cc:1511] Processing CreateTablet for tablet 8508c52406f8476ba4da651c8b8b7d37 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:45.002492 26365 tablet_service.cc:1511] Processing CreateTablet for tablet 04c9919a91214947966bdec9eb37aec4 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:45.004860 26364 tablet_service.cc:1511] Processing CreateTablet for tablet 37d87c825e49472bb270be7e0f277609 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:44.998042 26372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54584a4053b41c99b53dda055ba490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.011258 26370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b03910d4d549828b82cf0ea4fe938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.012650 26371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490b6ffef368438591943bc51cc81736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.013674 26369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c027ecfd3b4d3e889ef85da3520df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.014492 26366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8508c52406f8476ba4da651c8b8b7d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.016515 26365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c9919a91214947966bdec9eb37aec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.017311 26367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae45edb272b44a9aaaa37818f4c2ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.018070 26368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95a57fd9a10946479d74f32616f5437b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.022639 26364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d87c825e49472bb270be7e0f277609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.043015 26296 tablet_service.cc:1511] Processing CreateTablet for tablet c0b03910d4d549828b82cf0ea4fe938b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:45.046211 26295 tablet_service.cc:1511] Processing CreateTablet for tablet 43c027ecfd3b4d3e889ef85da3520df5 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:45.048996 26294 tablet_service.cc:1511] Processing CreateTablet for tablet 95a57fd9a10946479d74f32616f5437b (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:45.051757 26292 tablet_service.cc:1511] Processing CreateTablet for tablet 8508c52406f8476ba4da651c8b8b7d37 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:45.051757 26293 tablet_service.cc:1511] Processing CreateTablet for tablet cae45edb272b44a9aaaa37818f4c2ce7 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:45.052959 26296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b03910d4d549828b82cf0ea4fe938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.053969 26293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae45edb272b44a9aaaa37818f4c2ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.054773 26294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95a57fd9a10946479d74f32616f5437b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.055440 26292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8508c52406f8476ba4da651c8b8b7d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.061275 26295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c027ecfd3b4d3e889ef85da3520df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.070304 26291 tablet_service.cc:1511] Processing CreateTablet for tablet 04c9919a91214947966bdec9eb37aec4 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:45.072046 26291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c9919a91214947966bdec9eb37aec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.074281 26290 tablet_service.cc:1511] Processing CreateTablet for tablet 37d87c825e49472bb270be7e0f277609 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:45.074611 26217 tablet_service.cc:1511] Processing CreateTablet for tablet 37d87c825e49472bb270be7e0f277609 (DEFAULT_TABLE table=test-workload [id=2c859d2383ab4208bdcdab353423e69a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:45.075930 26290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d87c825e49472bb270be7e0f277609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.075930 26217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d87c825e49472bb270be7e0f277609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:45.126264 26423 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.129344 26422 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.139782 26423 tablet_bootstrap.cc:654] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.148098 26422 tablet_bootstrap.cc:654] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.153983 26424 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.169217 26422 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.170990 26422 ts_tablet_manager.cc:1403] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.010s
I20260709 13:07:45.171712 26424 tablet_bootstrap.cc:654] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.173375 26423 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.173985 26423 ts_tablet_manager.cc:1403] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.004s
I20260709 13:07:45.174584 26422 raft_consensus.cc:359] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.175820 26422 raft_consensus.cc:385] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.176203 26422 raft_consensus.cc:740] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.177201 26422 consensus_queue.cc:260] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.177840 26424 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.177747 26423 raft_consensus.cc:359] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.178581 26424 ts_tablet_manager.cc:1403] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 13:07:45.178841 26423 raft_consensus.cc:385] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.179332 26423 raft_consensus.cc:740] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.180979 26424 raft_consensus.cc:359] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.180625 26423 consensus_queue.cc:260] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.181625 26424 raft_consensus.cc:385] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.181986 26424 raft_consensus.cc:740] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.182828 26258 heartbeater.cc:499] Master 127.13.163.254:44849 was elected leader, sending a full tablet report...
I20260709 13:07:45.182912 26424 consensus_queue.cc:260] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.183884 26422 ts_tablet_manager.cc:1434] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.007s
I20260709 13:07:45.185449 26422 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.189543 26424 ts_tablet_manager.cc:1434] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.006s
I20260709 13:07:45.190351 26407 heartbeater.cc:499] Master 127.13.163.254:44849 was elected leader, sending a full tablet report...
I20260709 13:07:45.196038 26422 tablet_bootstrap.cc:654] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.199570 26424 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.206424 26333 heartbeater.cc:499] Master 127.13.163.254:44849 was elected leader, sending a full tablet report...
I20260709 13:07:45.208684 26424 tablet_bootstrap.cc:654] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.214201 26423 ts_tablet_manager.cc:1434] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.040s	user 0.021s	sys 0.012s
I20260709 13:07:45.215051 26422 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.215624 26423 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.215698 26422 ts_tablet_manager.cc:1403] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.014s
I20260709 13:07:45.219383 26422 raft_consensus.cc:359] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.220180 26422 raft_consensus.cc:385] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.220481 26422 raft_consensus.cc:740] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.221365 26424 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.221424 26422 consensus_queue.cc:260] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.222157 26424 ts_tablet_manager.cc:1403] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.003s
I20260709 13:07:45.224107 26422 ts_tablet_manager.cc:1434] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 13:07:45.224462 26423 tablet_bootstrap.cc:654] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.225265 26422 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.227150 26424 raft_consensus.cc:359] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.227960 26424 raft_consensus.cc:385] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.228331 26424 raft_consensus.cc:740] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.229257 26424 consensus_queue.cc:260] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.232894 26422 tablet_bootstrap.cc:654] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.238309 26424 ts_tablet_manager.cc:1434] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:07:45.240077 26424 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.241971 26423 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.242647 26423 ts_tablet_manager.cc:1403] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.009s
I20260709 13:07:45.246165 26423 raft_consensus.cc:359] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.247016 26423 raft_consensus.cc:385] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.247365 26423 raft_consensus.cc:740] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.248456 26423 consensus_queue.cc:260] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.250200 26424 tablet_bootstrap.cc:654] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.251236 26423 ts_tablet_manager.cc:1434] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:07:45.252470 26423 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.260011 26423 tablet_bootstrap.cc:654] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.266489 26422 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.267055 26422 ts_tablet_manager.cc:1403] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.017s
I20260709 13:07:45.269187 26424 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.270052 26424 ts_tablet_manager.cc:1403] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.021s
I20260709 13:07:45.270329 26422 raft_consensus.cc:359] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.271167 26422 raft_consensus.cc:385] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.271461 26422 raft_consensus.cc:740] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.272324 26422 consensus_queue.cc:260] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.273097 26423 tablet_bootstrap.cc:492] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.273803 26423 ts_tablet_manager.cc:1403] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:07:45.273650 26424 raft_consensus.cc:359] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.274526 26424 raft_consensus.cc:385] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.274588 26422 ts_tablet_manager.cc:1434] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.002s
I20260709 13:07:45.274993 26424 raft_consensus.cc:740] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.275772 26422 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.275913 26424 consensus_queue.cc:260] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.277506 26423 raft_consensus.cc:359] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.278422 26423 raft_consensus.cc:385] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.278620 26424 ts_tablet_manager.cc:1434] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:45.278759 26423 raft_consensus.cc:740] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.279910 26424 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.279834 26423 consensus_queue.cc:260] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.282716 26423 ts_tablet_manager.cc:1434] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:45.283293 26422 tablet_bootstrap.cc:654] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.283800 26423 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.289028 26424 tablet_bootstrap.cc:654] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.296080 26423 tablet_bootstrap.cc:654] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.303313 26424 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.303917 26424 ts_tablet_manager.cc:1403] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.008s
I20260709 13:07:45.308362 26424 raft_consensus.cc:359] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.309449 26424 raft_consensus.cc:385] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.309777 26424 raft_consensus.cc:740] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.310714 26424 consensus_queue.cc:260] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.312376 26423 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.313023 26423 ts_tablet_manager.cc:1403] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.006s
I20260709 13:07:45.314251 26422 tablet_bootstrap.cc:492] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.314850 26422 ts_tablet_manager.cc:1403] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.016s
I20260709 13:07:45.314911 26424 ts_tablet_manager.cc:1434] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:07:45.316187 26424 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.316502 26423 raft_consensus.cc:359] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.317407 26423 raft_consensus.cc:385] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.317759 26423 raft_consensus.cc:740] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.318023 26422 raft_consensus.cc:359] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.318826 26422 raft_consensus.cc:385] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.319247 26422 raft_consensus.cc:740] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.318861 26423 consensus_queue.cc:260] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.320155 26422 consensus_queue.cc:260] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.324311 26424 tablet_bootstrap.cc:654] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.325263 26422 ts_tablet_manager.cc:1434] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:07:45.326426 26422 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.326555 26423 ts_tablet_manager.cc:1434] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:07:45.327924 26423 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.336663 26422 tablet_bootstrap.cc:654] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.337556 26423 tablet_bootstrap.cc:654] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.344973 26429 raft_consensus.cc:493] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.345777 26429 raft_consensus.cc:515] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.348819 26424 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.349677 26424 ts_tablet_manager.cc:1403] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.011s
I20260709 13:07:45.354020 26424 raft_consensus.cc:359] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.355028 26429 leader_election.cc:290] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.355039 26424 raft_consensus.cc:385] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.356410 26424 raft_consensus.cc:740] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.357717 26424 consensus_queue.cc:260] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.365048 26424 ts_tablet_manager.cc:1434] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:07:45.366515 26424 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
W20260709 13:07:45.370379 26178 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:45.372350 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
W20260709 13:07:45.374536 26272 leader_election.cc:343] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.379333 26424 tablet_bootstrap.cc:654] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.379376 26423 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.381222 26423 ts_tablet_manager.cc:1403] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.006s
I20260709 13:07:45.382185 26422 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.382920 26422 ts_tablet_manager.cc:1403] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.057s	user 0.039s	sys 0.010s
I20260709 13:07:45.384594 26423 raft_consensus.cc:359] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.385522 26423 raft_consensus.cc:385] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.385910 26423 raft_consensus.cc:740] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.386868 26422 raft_consensus.cc:359] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.386937 26423 consensus_queue.cc:260] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.387902 26422 raft_consensus.cc:385] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.388350 26422 raft_consensus.cc:740] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.389402 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.389357 26422 consensus_queue.cc:260] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
W20260709 13:07:45.391168 26271 leader_election.cc:343] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.391631 26271 leader_election.cc:304] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39
I20260709 13:07:45.392506 26429 raft_consensus.cc:2749] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:45.394814 26423 ts_tablet_manager.cc:1434] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:07:45.395148 26422 ts_tablet_manager.cc:1434] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:07:45.396306 26423 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.396641 26422 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.404685 26423 tablet_bootstrap.cc:654] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.406304 26424 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.406862 26424 ts_tablet_manager.cc:1403] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.006s
I20260709 13:07:45.407131 26422 tablet_bootstrap.cc:654] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.410697 26424 raft_consensus.cc:359] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.411803 26424 raft_consensus.cc:385] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.412194 26424 raft_consensus.cc:740] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.413408 26424 consensus_queue.cc:260] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.418960 26422 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.418989 26423 tablet_bootstrap.cc:492] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.419166 26424 ts_tablet_manager.cc:1434] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:07:45.419746 26422 ts_tablet_manager.cc:1403] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:07:45.420037 26423 ts_tablet_manager.cc:1403] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.001s
I20260709 13:07:45.420827 26424 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.423170 26422 raft_consensus.cc:359] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.424234 26422 raft_consensus.cc:385] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.423806 26423 raft_consensus.cc:359] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.424561 26422 raft_consensus.cc:740] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.424696 26423 raft_consensus.cc:385] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.425218 26423 raft_consensus.cc:740] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.425761 26422 consensus_queue.cc:260] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.426358 26423 consensus_queue.cc:260] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.429527 26423 ts_tablet_manager.cc:1434] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:07:45.429647 26424 tablet_bootstrap.cc:654] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.430436 26422 ts_tablet_manager.cc:1434] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:07:45.431387 26423 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.431959 26428 raft_consensus.cc:493] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.432505 26428 raft_consensus.cc:515] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.438361 26428 leader_election.cc:290] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.431607 26422 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.447443 26429 raft_consensus.cc:493] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.447986 26430 raft_consensus.cc:493] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.448388 26422 tablet_bootstrap.cc:654] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.448096 26429 raft_consensus.cc:515] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.448583 26430 raft_consensus.cc:515] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.450803 26429 leader_election.cc:290] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.453184 26430 leader_election.cc:290] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.454526 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.455503 26382 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:45.457365 26271 leader_election.cc:304] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.458415 26429 raft_consensus.cc:2804] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.458804 26429 raft_consensus.cc:493] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.459131 26429 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.460268 26423 tablet_bootstrap.cc:654] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.461382 26422 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.462138 26422 ts_tablet_manager.cc:1403] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.004s
I20260709 13:07:45.464403 26424 tablet_bootstrap.cc:492] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.466123 26422 raft_consensus.cc:359] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.467075 26422 raft_consensus.cc:385] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.467500 26422 raft_consensus.cc:740] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.467430 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.468429 26233 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:45.468703 26422 consensus_queue.cc:260] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.470144 26424 ts_tablet_manager.cc:1403] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.003s
I20260709 13:07:45.471928 26422 ts_tablet_manager.cc:1434] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:07:45.473296 26422 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.481487 26422 tablet_bootstrap.cc:654] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.481848 26424 raft_consensus.cc:359] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.482728 26424 raft_consensus.cc:385] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.483042 26424 raft_consensus.cc:740] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.483814 26424 consensus_queue.cc:260] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.496295 26429 raft_consensus.cc:515] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.496894 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.497599 26233 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.499725 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.500342 26382 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.501382 26424 ts_tablet_manager.cc:1434] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:07:45.502534 26424 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.505959 26423 tablet_bootstrap.cc:492] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.505885 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490b6ffef368438591943bc51cc81736" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.506618 26423 ts_tablet_manager.cc:1403] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.075s	user 0.007s	sys 0.006s
I20260709 13:07:45.506683 26308 raft_consensus.cc:2468] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.508596 26196 leader_election.cc:304] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.508939 26429 leader_election.cc:290] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.509747 26424 tablet_bootstrap.cc:654] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.510313 26428 raft_consensus.cc:2804] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.510957 26428 raft_consensus.cc:493] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.511435 26428 raft_consensus.cc:3060] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.499994 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.513422 26233 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.524986 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490b6ffef368438591943bc51cc81736" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.526000 26424 tablet_bootstrap.cc:492] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.526579 26424 ts_tablet_manager.cc:1403] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.014s
W20260709 13:07:45.526772 26195 leader_election.cc:343] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.529551 26424 raft_consensus.cc:359] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.530408 26424 raft_consensus.cc:385] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.529778 26428 raft_consensus.cc:515] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.530719 26424 raft_consensus.cc:740] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.531649 26424 consensus_queue.cc:260] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.521063 26346 leader_election.cc:304] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.534032 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490b6ffef368438591943bc51cc81736" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.534898 26308 raft_consensus.cc:3060] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.536963 26424 ts_tablet_manager.cc:1434] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:07:45.538086 26424 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:45.543174 26430 raft_consensus.cc:2804] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.543681 26430 raft_consensus.cc:493] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.544102 26424 tablet_bootstrap.cc:654] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.544090 26430 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.546412 26308 raft_consensus.cc:2468] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
W20260709 13:07:45.544510 26346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:45401, user_credentials={real_user=slave}} blocked reactor thread for 45609us
I20260709 13:07:45.547951 26196 leader_election.cc:304] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.553177 26382 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:45.526758 26423 raft_consensus.cc:359] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.555219 26423 raft_consensus.cc:385] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.555104 26430 raft_consensus.cc:515] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.555634 26423 raft_consensus.cc:740] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.556460 26423 consensus_queue.cc:260] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.557508 26271 leader_election.cc:304] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.559443 26423 ts_tablet_manager.cc:1434] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.002s
I20260709 13:07:45.559810 26429 raft_consensus.cc:2804] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.560582 26429 raft_consensus.cc:697] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:45.562085 26459 raft_consensus.cc:2804] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.561985 26429 consensus_queue.cc:237] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.564347 26458 raft_consensus.cc:493] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.565115 26458 raft_consensus.cc:515] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.567289 26423 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.532539 26233 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:45.582082 26423 tablet_bootstrap.cc:654] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.588814 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.589433 26458 leader_election.cc:290] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.590142 26458 raft_consensus.cc:493] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.591095 26458 raft_consensus.cc:515] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.591758 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.592561 26232 raft_consensus.cc:2468] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.589560 26233 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.593848 26458 leader_election.cc:290] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.589814 26465 raft_consensus.cc:493] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.595468 26465 raft_consensus.cc:515] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.597033 26430 leader_election.cc:290] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.599532 26465 leader_election.cc:290] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.600660 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.603193 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490b6ffef368438591943bc51cc81736" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.603863 26381 raft_consensus.cc:3060] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.604060 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.604290 26428 leader_election.cc:290] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
W20260709 13:07:45.605918 26271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.195:45787, user_credentials={real_user=slave}} blocked reactor thread for 48531.8us
I20260709 13:07:45.606434 26346 leader_election.cc:304] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.607381 26429 raft_consensus.cc:493] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.607846 26307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.608140 26430 raft_consensus.cc:2804] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.601063 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.608989 26305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.608183 26429 raft_consensus.cc:515] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.609241 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.609171 26382 raft_consensus.cc:2468] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.610052 26305 raft_consensus.cc:2468] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.610278 26307 raft_consensus.cc:2468] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.610975 26195 leader_election.cc:304] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.598419 26129 catalog_manager.cc:5697] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 0 to 1, leader changed from <none> to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.612417 26465 raft_consensus.cc:493] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.610277 26306 raft_consensus.cc:2468] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.613145 26465 raft_consensus.cc:515] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.614975 26381 raft_consensus.cc:2468] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.615991 26429 leader_election.cc:290] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.616288 26465 leader_election.cc:290] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.617105 26465 raft_consensus.cc:2804] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.616864 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.616823 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.617669 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54584a4053b41c99b53dda055ba490d" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.617865 26465 raft_consensus.cc:493] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.617938 26232 raft_consensus.cc:2468] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:45.618412 26382 raft_consensus.cc:2468] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.618903 26465 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.619877 26272 leader_election.cc:304] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.620051 26195 leader_election.cc:304] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.622259 26428 raft_consensus.cc:493] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.622818 26428 raft_consensus.cc:515] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.622776 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
W20260709 13:07:45.607498 26346 leader_election.cc:343] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.608634 26430 raft_consensus.cc:493] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:45.628939 26346 leader_election.cc:343] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.630373 26346 leader_election.cc:304] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: e600c189c03f439abeed018164fc6d39
I20260709 13:07:45.629285 26430 raft_consensus.cc:3060] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.631011 26422 tablet_bootstrap.cc:492] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.631695 26422 ts_tablet_manager.cc:1403] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.159s	user 0.049s	sys 0.049s
I20260709 13:07:45.632583 26423 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.633137 26423 ts_tablet_manager.cc:1403] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.073s	user 0.017s	sys 0.000s
I20260709 13:07:45.626384 26429 raft_consensus.cc:2804] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:45.636626 26346 leader_election.cc:343] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:45.633793 26459 raft_consensus.cc:697] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:45.637529 26307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.634629 26429 raft_consensus.cc:493] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.638375 26423 raft_consensus.cc:359] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.641511 26423 raft_consensus.cc:385] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.641774 26423 raft_consensus.cc:740] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.639214 26422 raft_consensus.cc:359] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.635277 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54584a4053b41c99b53dda055ba490d" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.633306 26458 raft_consensus.cc:2804] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.642215 26422 raft_consensus.cc:385] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.644168 26458 raft_consensus.cc:493] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.644433 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.644562 26422 raft_consensus.cc:740] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.642534 26307 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.645352 26382 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.644665 26458 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.645900 26422 consensus_queue.cc:260] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.643288 26459 consensus_queue.cc:237] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.642735 26429 raft_consensus.cc:3060] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.646752 26195 leader_election.cc:304] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.636648 26465 raft_consensus.cc:515] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.643262 26308 raft_consensus.cc:2468] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:45.642658 26428 leader_election.cc:290] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.657497 26423 consensus_queue.cc:260] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.664327 26424 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:45.664983 26424 ts_tablet_manager.cc:1403] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.127s	user 0.006s	sys 0.017s
I20260709 13:07:45.666486 26423 ts_tablet_manager.cc:1434] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:07:45.668872 26423 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:45.671903 26422 ts_tablet_manager.cc:1434] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:07:45.672904 26422 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:45.669977 26428 raft_consensus.cc:2804] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.679098 26233 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:45.679229 26469 raft_consensus.cc:493] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.680172 26469 raft_consensus.cc:515] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.682169 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.678164 26428 raft_consensus.cc:493] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.683406 26428 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.684535 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.685207 26382 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.686724 26429 raft_consensus.cc:515] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.675262 26458 raft_consensus.cc:515] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.689827 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.690486 26232 raft_consensus.cc:3060] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.680532 26346 leader_election.cc:304] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:45.681133 26465 leader_election.cc:290] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.694232 26463 raft_consensus.cc:2804] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.694003 26430 raft_consensus.cc:515] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.695398 26381 raft_consensus.cc:2393] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:45.696349 26430 leader_election.cc:290] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.682811 26308 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.697620 26230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.698172 26458 leader_election.cc:290] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.699102 26307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.699932 26307 raft_consensus.cc:2393] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a31357e1e0cc4058bb24dc26e0aa2522 in current term 1: Already voted for candidate e72cb39b26f7438abc76d6f96e894597 in this term.
I20260709 13:07:45.696214 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.701153 26422 tablet_bootstrap.cc:654] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.701967 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.702571 26306 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.704335 26429 leader_election.cc:290] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.705525 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8508c52406f8476ba4da651c8b8b7d37" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.706620 26381 raft_consensus.cc:2393] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:45.707743 26469 leader_election.cc:290] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.709043 26231 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.709383 26423 tablet_bootstrap.cc:654] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:45.717487 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.718050 26346 leader_election.cc:304] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522; no voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:45.719010 26381 raft_consensus.cc:2393] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e600c189c03f439abeed018164fc6d39 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:45.719147 26430 raft_consensus.cc:2749] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:45.706341 26424 raft_consensus.cc:359] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.709843 26305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.692942 26462 raft_consensus.cc:2804] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.725070 26462 raft_consensus.cc:493] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.725680 26462 raft_consensus.cc:3060] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.726691 26428 raft_consensus.cc:515] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.729031 26428 leader_election.cc:290] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.730724 26306 raft_consensus.cc:2468] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:45.731938 26346 leader_election.cc:304] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.732769 26308 raft_consensus.cc:2468] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.732790 26430 raft_consensus.cc:2804] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.733954 26430 raft_consensus.cc:697] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:45.733243 26196 leader_election.cc:304] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e600c189c03f439abeed018164fc6d39; no voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:45.734965 26430 consensus_queue.cc:237] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.735656 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.736409 26308 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.741366 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.742334 26381 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.745154 26232 raft_consensus.cc:2468] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:45.723627 26424 raft_consensus.cc:385] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.746115 26196 leader_election.cc:304] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.749380 26424 raft_consensus.cc:740] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.747051 26382 raft_consensus.cc:2468] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.750274 26272 leader_election.cc:304] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:45.751892 26429 raft_consensus.cc:2804] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.752599 26429 raft_consensus.cc:697] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:45.753746 26463 raft_consensus.cc:697] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:45.753719 26429 consensus_queue.cc:237] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.755225 26463 consensus_queue.cc:237] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.750341 26459 raft_consensus.cc:2804] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.759572 26459 raft_consensus.cc:697] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:45.760622 26459 consensus_queue.cc:237] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.750703 26424 consensus_queue.cc:260] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.747983 26231 raft_consensus.cc:2468] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:45.766479 26470 raft_consensus.cc:2749] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:45.768947 26424 ts_tablet_manager.cc:1434] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.000s
I20260709 13:07:45.775367 26127 catalog_manager.cc:5697] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.784869 26462 raft_consensus.cc:515] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.790469 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54584a4053b41c99b53dda055ba490d" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:45.791180 26382 raft_consensus.cc:3060] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.791975 26308 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.791507 26129 catalog_manager.cc:5697] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 0 to 1, leader changed from <none> to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.794140 26381 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.792821 26423 tablet_bootstrap.cc:492] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:45.797575 26196 leader_election.cc:304] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:45.800122 26422 tablet_bootstrap.cc:492] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:45.803930 26422 ts_tablet_manager.cc:1403] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.131s	user 0.016s	sys 0.036s
I20260709 13:07:45.805640 26127 catalog_manager.cc:5697] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.803756 26470 raft_consensus.cc:2804] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.807237 26422 raft_consensus.cc:359] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.807816 26470 raft_consensus.cc:697] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:45.808012 26422 raft_consensus.cc:385] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.808319 26422 raft_consensus.cc:740] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.808750 26470 consensus_queue.cc:237] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.809049 26422 consensus_queue.cc:260] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.808522 26128 catalog_manager.cc:5697] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:45.811357 26422 ts_tablet_manager.cc:1434] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:45.800657 26423 ts_tablet_manager.cc:1403] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.022s
I20260709 13:07:45.815992 26382 raft_consensus.cc:2468] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.817478 26462 leader_election.cc:290] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.817534 26195 leader_election.cc:304] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.818904 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54584a4053b41c99b53dda055ba490d" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.819718 26308 raft_consensus.cc:3060] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.820753 26462 raft_consensus.cc:2804] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:45.817571 26423 raft_consensus.cc:359] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.821422 26462 raft_consensus.cc:697] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:45.821631 26423 raft_consensus.cc:385] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:45.821943 26423 raft_consensus.cc:740] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.822700 26423 consensus_queue.cc:260] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.822358 26462 consensus_queue.cc:237] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } }
I20260709 13:07:45.825106 26423 ts_tablet_manager.cc:1434] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:45.838271 26308 raft_consensus.cc:2468] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:45.838960 26429 raft_consensus.cc:493] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.839541 26429 raft_consensus.cc:515] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.841744 26429 leader_election.cc:290] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:45.843336 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.843879 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:45.844313 26231 raft_consensus.cc:2393] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:45.846032 26272 leader_election.cc:304] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39
I20260709 13:07:45.845826 26126 catalog_manager.cc:5697] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.847306 26429 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.847949 26126 catalog_manager.cc:5697] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.852361 26429 raft_consensus.cc:2749] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:45.856549 26129 catalog_manager.cc:5697] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:45.978207 26463 raft_consensus.cc:493] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.978719 26463 raft_consensus.cc:515] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.980444 26463 leader_election.cc:290] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.981537 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d87c825e49472bb270be7e0f277609" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:45.981906 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d87c825e49472bb270be7e0f277609" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:45.982259 26231 raft_consensus.cc:2468] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.982589 26308 raft_consensus.cc:2468] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:45.983441 26346 leader_election.cc:304] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:45.984200 26463 raft_consensus.cc:2804] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:45.984560 26463 raft_consensus.cc:493] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.984942 26463 raft_consensus.cc:3060] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.990402 26463 raft_consensus.cc:515] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:45.992483 26463 leader_election.cc:290] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:45.993252 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d87c825e49472bb270be7e0f277609" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:45.993500 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d87c825e49472bb270be7e0f277609" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:45.993896 26231 raft_consensus.cc:3060] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.994119 26308 raft_consensus.cc:3060] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:45.999853 26231 raft_consensus.cc:2468] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:46.000532 26308 raft_consensus.cc:2468] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:46.001237 26346 leader_election.cc:304] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:46.002079 26463 raft_consensus.cc:2804] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:46.002578 26463 raft_consensus.cc:697] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:46.003271 26463 consensus_queue.cc:237] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.011545 26129 catalog_manager.cc:5697] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:46.054469 26429 consensus_queue.cc:1048] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.070998 26429 consensus_queue.cc:1048] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.098681 26308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
mode: GRACEFUL
new_leader_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:34882
I20260709 13:07:46.099440 26308 raft_consensus.cc:606] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Received request to transfer leadership to TS e600c189c03f439abeed018164fc6d39
I20260709 13:07:46.117668 26382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
mode: GRACEFUL
new_leader_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:34602
I20260709 13:07:46.118312 26382 raft_consensus.cc:606] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Received request to transfer leadership to TS e600c189c03f439abeed018164fc6d39
I20260709 13:07:46.136723 26231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:07:46.137439 26231 raft_consensus.cc:606] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:46.138958 26463 consensus_queue.cc:1048] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.143968 26233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:07:46.144486 26233 raft_consensus.cc:606] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:46.148372 26462 consensus_queue.cc:1048] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.154373 26458 consensus_queue.cc:1048] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.156251 26381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34602
I20260709 13:07:46.157109 26381 raft_consensus.cc:606] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:46.169111 26428 consensus_queue.cc:1048] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.172577 26430 raft_consensus.cc:993] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.173177 26430 raft_consensus.cc:1081] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.175957 26305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34876
I20260709 13:07:46.176661 26305 raft_consensus.cc:493] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.177491 26305 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.184706 26305 raft_consensus.cc:515] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.187552 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:46.187894 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c9919a91214947966bdec9eb37aec4" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:46.188212 26233 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.188566 26381 raft_consensus.cc:3055] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.188912 26381 raft_consensus.cc:740] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:46.189827 26461 consensus_queue.cc:1048] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:46.189875 26381 consensus_queue.cc:260] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.192777 26381 raft_consensus.cc:3060] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.195879 26233 raft_consensus.cc:2468] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:46.197468 26462 consensus_queue.cc:1048] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.197789 26272 leader_election.cc:304] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:46.199965 26461 raft_consensus.cc:2804] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.200911 26305 leader_election.cc:290] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:46.206336 26381 raft_consensus.cc:2468] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:46.207739 26461 raft_consensus.cc:697] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:46.208813 26461 consensus_queue.cc:237] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.209419 26428 raft_consensus.cc:993] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.210080 26469 raft_consensus.cc:1081] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.212158 26305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34860
I20260709 13:07:46.221433 26462 consensus_queue.cc:1048] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:46.226827 26305 raft_consensus.cc:493] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.227550 26305 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.227429 26500 consensus_queue.cc:1048] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:46.237923 26305 raft_consensus.cc:515] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.240399 26458 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:46.243680 26305 leader_election.cc:290] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:46.243891 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:46.244575 26231 raft_consensus.cc:3055] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.244925 26231 raft_consensus.cc:740] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:46.245782 26231 consensus_queue.cc:260] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.246405 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae45edb272b44a9aaaa37818f4c2ce7" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:46.246937 26231 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.254104 26231 raft_consensus.cc:2468] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:46.255478 26272 leader_election.cc:304] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:46.256335 26500 raft_consensus.cc:2804] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.256829 26500 raft_consensus.cc:697] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:46.257622 26500 consensus_queue.cc:237] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.253926 26126 catalog_manager.cc:5697] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.262495 26430 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.270094 26430 raft_consensus.cc:993] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: : Instructing follower e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:46.270619 26430 raft_consensus.cc:1081] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Signalling peer e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:46.272127 26231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:47462
I20260709 13:07:46.272612 26231 raft_consensus.cc:493] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.272949 26231 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.278456 26231 raft_consensus.cc:515] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.280433 26231 leader_election.cc:290] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:46.281646 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:46.282325 26382 raft_consensus.cc:3055] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.282665 26382 raft_consensus.cc:740] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:46.283505 26382 consensus_queue.cc:260] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.284693 26382 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.285028 26305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:46.285657 26305 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.284675 26127 catalog_manager.cc:5697] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from e600c189c03f439abeed018164fc6d39 (127.13.163.193) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.291496 26305 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:46.292729 26196 leader_election.cc:304] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:46.292814 26382 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:46.293627 26462 raft_consensus.cc:2804] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.294129 26462 raft_consensus.cc:697] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:46.294884 26462 consensus_queue.cc:237] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.302994 26126 catalog_manager.cc:5697] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 1 to 2, leader changed from a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195) to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.311964 26462 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.321435 26469 raft_consensus.cc:993] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.321991 26459 raft_consensus.cc:1081] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:46.324043 26306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34860
I20260709 13:07:46.324656 26306 raft_consensus.cc:493] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.325076 26306 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.328963 26462 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:46.332805 26306 raft_consensus.cc:515] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.336266 26306 leader_election.cc:290] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:46.336948 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:46.337414 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:46.337643 26231 raft_consensus.cc:3055] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.338101 26231 raft_consensus.cc:740] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:46.338927 26231 consensus_queue.cc:260] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.340185 26231 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.347623 26231 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:46.348783 26272 leader_election.cc:304] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:46.349644 26500 raft_consensus.cc:2804] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.350232 26500 raft_consensus.cc:697] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:46.351060 26500 consensus_queue.cc:237] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.358024 26127 catalog_manager.cc:5697] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from e600c189c03f439abeed018164fc6d39 (127.13.163.193) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.417555 26462 consensus_queue.cc:1048] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.428498 26459 consensus_queue.cc:1048] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.463888 26497 raft_consensus.cc:993] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: : Instructing follower e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:46.464335 26500 raft_consensus.cc:1081] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Signalling peer e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:46.465860 26231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:47450
I20260709 13:07:46.466393 26231 raft_consensus.cc:493] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.466789 26231 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.468552 26458 consensus_queue.cc:1048] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.473752 26231 raft_consensus.cc:515] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.476717 26231 leader_election.cc:290] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:46.477316 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:46.477964 26382 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.476475 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b03910d4d549828b82cf0ea4fe938b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:46.479178 26306 raft_consensus.cc:3055] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.480415 26306 raft_consensus.cc:740] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:46.481150 26458 consensus_queue.cc:1048] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:46.481628 26306 consensus_queue.cc:260] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.483016 26306 raft_consensus.cc:3060] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.491127 26382 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:46.494478 26306 raft_consensus.cc:2468] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:46.494630 26195 leader_election.cc:304] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:46.495728 26469 raft_consensus.cc:2804] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.496218 26469 raft_consensus.cc:697] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:46.497064 26469 consensus_queue.cc:237] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:46.504160 26127 catalog_manager.cc:5697] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 1 to 2, leader changed from e72cb39b26f7438abc76d6f96e894597 (127.13.163.194) to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.659952 26231 raft_consensus.cc:1275] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.661382 26500 consensus_queue.cc:1048] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.668956 26382 raft_consensus.cc:1275] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.670848 26497 consensus_queue.cc:1048] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.684808 26231 raft_consensus.cc:1275] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.686151 26497 consensus_queue.cc:1048] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.694571 26382 raft_consensus.cc:3060] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.702854 26382 raft_consensus.cc:1275] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.704146 26429 consensus_queue.cc:1048] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.768942 26306 raft_consensus.cc:1275] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.770318 26469 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.778666 26382 raft_consensus.cc:1275] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.780391 26469 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.816984 26231 raft_consensus.cc:1275] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.818365 26429 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.826025 26382 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.833953 26382 raft_consensus.cc:1275] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.835270 26500 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.904188 26382 raft_consensus.cc:1275] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.905654 26469 consensus_queue.cc:1048] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.913311 26306 raft_consensus.cc:1275] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.915004 26462 consensus_queue.cc:1048] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:49.195465 26127 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48188:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:49.197829 26127 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:49.252651 26216 tablet_service.cc:1511] Processing CreateTablet for tablet caf6f070920e4604b451af2f70d02962 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:49.254009 26216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf6f070920e4604b451af2f70d02962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.253775 26290 tablet_service.cc:1511] Processing CreateTablet for tablet caf6f070920e4604b451af2f70d02962 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:49.255110 26364 tablet_service.cc:1511] Processing CreateTablet for tablet caf6f070920e4604b451af2f70d02962 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:49.256404 26364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf6f070920e4604b451af2f70d02962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.257503 26295 tablet_service.cc:1511] Processing CreateTablet for tablet 102ccba0b0614ab9bad959c1d1b3df3e (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:49.258992 26217 tablet_service.cc:1511] Processing CreateTablet for tablet 102ccba0b0614ab9bad959c1d1b3df3e (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:49.260296 26217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102ccba0b0614ab9bad959c1d1b3df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.260779 26218 tablet_service.cc:1511] Processing CreateTablet for tablet 33a44bfdc7b144878d22681831f1aeb9 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:49.262145 26218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a44bfdc7b144878d22681831f1aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.262388 26294 tablet_service.cc:1511] Processing CreateTablet for tablet 33a44bfdc7b144878d22681831f1aeb9 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:49.255115 26290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf6f070920e4604b451af2f70d02962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.265690 26295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102ccba0b0614ab9bad959c1d1b3df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.265882 26371 tablet_service.cc:1511] Processing CreateTablet for tablet 33a44bfdc7b144878d22681831f1aeb9 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:49.265017 26367 tablet_service.cc:1511] Processing CreateTablet for tablet 102ccba0b0614ab9bad959c1d1b3df3e (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:49.267234 26371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a44bfdc7b144878d22681831f1aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.267500 26294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a44bfdc7b144878d22681831f1aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.269788 26367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102ccba0b0614ab9bad959c1d1b3df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.278529 26372 tablet_service.cc:1511] Processing CreateTablet for tablet 32538c59d81b4409b0b8f10cd66f9057 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:49.280015 26372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32538c59d81b4409b0b8f10cd66f9057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.282209 26366 tablet_service.cc:1511] Processing CreateTablet for tablet 2842c695cdfb4ab8b762d80ec279cac2 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:49.282557 26219 tablet_service.cc:1511] Processing CreateTablet for tablet 32538c59d81b4409b0b8f10cd66f9057 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:49.283670 26366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2842c695cdfb4ab8b762d80ec279cac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.283751 26219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32538c59d81b4409b0b8f10cd66f9057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.288097 26296 tablet_service.cc:1511] Processing CreateTablet for tablet 32538c59d81b4409b0b8f10cd66f9057 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:49.289681 26296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32538c59d81b4409b0b8f10cd66f9057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.296066 26523 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.299038 26298 tablet_service.cc:1511] Processing CreateTablet for tablet 2842c695cdfb4ab8b762d80ec279cac2 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:49.299655 26223 tablet_service.cc:1511] Processing CreateTablet for tablet 2842c695cdfb4ab8b762d80ec279cac2 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:49.304800 26298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2842c695cdfb4ab8b762d80ec279cac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.304920 26223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2842c695cdfb4ab8b762d80ec279cac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.310722 26523 tablet_bootstrap.cc:654] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.344650 26524 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.360351 26218 tablet_service.cc:1511] Processing CreateTablet for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:49.361861 26218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.362003 26296 tablet_service.cc:1511] Processing CreateTablet for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:49.363354 26296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.368969 26372 tablet_service.cc:1511] Processing CreateTablet for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:49.370648 26372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d3b4f5c4294330bb8f10ca2a7c80b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.383559 26524 tablet_bootstrap.cc:654] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.390002 26526 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.392321 26298 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0568319830424f82a86cfda26b39bb (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:49.392509 26371 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0568319830424f82a86cfda26b39bb (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:49.393757 26371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0568319830424f82a86cfda26b39bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.394970 26523 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.395601 26523 ts_tablet_manager.cc:1403] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.100s	user 0.033s	sys 0.015s
I20260709 13:07:49.398886 26523 raft_consensus.cc:359] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.400363 26219 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0568319830424f82a86cfda26b39bb (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:49.417922 26219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0568319830424f82a86cfda26b39bb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:49.407927 26178 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:49.408538 26296 tablet_service.cc:1511] Processing CreateTablet for tablet 4846873ec5e549d385583ce30767e79b (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:49.411315 26216 tablet_service.cc:1511] Processing CreateTablet for tablet ac6540caac4247bf8f6b3a36418ad473 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:49.419633 26216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6540caac4247bf8f6b3a36418ad473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.422480 26524 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.423787 26524 ts_tablet_manager.cc:1403] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.079s	user 0.005s	sys 0.011s
I20260709 13:07:49.412424 26364 tablet_service.cc:1511] Processing CreateTablet for tablet 4846873ec5e549d385583ce30767e79b (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:49.411350 26217 tablet_service.cc:1511] Processing CreateTablet for tablet 4846873ec5e549d385583ce30767e79b (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:49.425952 26217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4846873ec5e549d385583ce30767e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.414824 26371 tablet_service.cc:1511] Processing CreateTablet for tablet ac6540caac4247bf8f6b3a36418ad473 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:49.415114 26295 tablet_service.cc:1511] Processing CreateTablet for tablet ac6540caac4247bf8f6b3a36418ad473 (DEFAULT_TABLE table=soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:49.416378 26526 tablet_bootstrap.cc:654] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.427625 26371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6540caac4247bf8f6b3a36418ad473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.430516 26524 raft_consensus.cc:359] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.431665 26524 raft_consensus.cc:385] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.431965 26524 raft_consensus.cc:740] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.393757 26298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0568319830424f82a86cfda26b39bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.432782 26524 consensus_queue.cc:260] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.433494 26295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6540caac4247bf8f6b3a36418ad473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.435359 26296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4846873ec5e549d385583ce30767e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.438259 26524 ts_tablet_manager.cc:1434] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:07:49.438864 26364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4846873ec5e549d385583ce30767e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:49.439601 26524 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.446740 26524 tablet_bootstrap.cc:654] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.451525 26523 raft_consensus.cc:385] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.451941 26523 raft_consensus.cc:740] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.455242 26523 consensus_queue.cc:260] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.462386 26523 ts_tablet_manager.cc:1434] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.001s
I20260709 13:07:49.463433 26523 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.470525 26526 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.471112 26526 ts_tablet_manager.cc:1403] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.000s
I20260709 13:07:49.486752 26526 raft_consensus.cc:359] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.481710 26523 tablet_bootstrap.cc:654] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.487668 26526 raft_consensus.cc:385] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.490948 26526 raft_consensus.cc:740] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.491814 26526 consensus_queue.cc:260] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.494891 26526 ts_tablet_manager.cc:1434] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:07:49.496147 26526 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.501163 26523 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.501773 26523 ts_tablet_manager.cc:1403] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.001s
I20260709 13:07:49.503365 26526 tablet_bootstrap.cc:654] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.504733 26523 raft_consensus.cc:359] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.505616 26523 raft_consensus.cc:385] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.505915 26523 raft_consensus.cc:740] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.506489 26524 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.507195 26524 ts_tablet_manager.cc:1403] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.010s
I20260709 13:07:49.506784 26523 consensus_queue.cc:260] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.511498 26523 ts_tablet_manager.cc:1434] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:07:49.512559 26523 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.512125 26524 raft_consensus.cc:359] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.513245 26524 raft_consensus.cc:385] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.513656 26524 raft_consensus.cc:740] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.515041 26524 consensus_queue.cc:260] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.520779 26524 ts_tablet_manager.cc:1434] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:07:49.520898 26523 tablet_bootstrap.cc:654] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.521822 26524 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.527864 26523 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.526248 26526 tablet_bootstrap.cc:492] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.531044 26524 tablet_bootstrap.cc:654] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.531793 26523 ts_tablet_manager.cc:1403] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.001s
I20260709 13:07:49.533061 26526 ts_tablet_manager.cc:1403] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.011s
I20260709 13:07:49.535234 26523 raft_consensus.cc:359] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.535882 26523 raft_consensus.cc:385] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.535831 26526 raft_consensus.cc:359] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.536401 26523 raft_consensus.cc:740] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.536620 26526 raft_consensus.cc:385] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.537158 26526 raft_consensus.cc:740] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.537520 26524 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.537266 26523 consensus_queue.cc:260] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.538064 26524 ts_tablet_manager.cc:1403] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.005s
I20260709 13:07:49.537884 26526 consensus_queue.cc:260] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.539832 26523 ts_tablet_manager.cc:1434] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:49.540926 26523 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.541491 26462 raft_consensus.cc:493] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.541364 26524 raft_consensus.cc:359] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.542187 26524 raft_consensus.cc:385] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.542047 26462 raft_consensus.cc:515] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.542506 26524 raft_consensus.cc:740] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.543356 26524 consensus_queue.cc:260] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.544031 26526 ts_tablet_manager.cc:1434] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:07:49.545656 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a44bfdc7b144878d22681831f1aeb9" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.546149 26526 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.546351 26381 raft_consensus.cc:2468] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:49.547887 26524 ts_tablet_manager.cc:1434] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.007s
I20260709 13:07:49.548381 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a44bfdc7b144878d22681831f1aeb9" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:49.549098 26524 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.551486 26523 tablet_bootstrap.cc:654] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.554793 26195 leader_election.cc:304] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
W20260709 13:07:49.556316 26196 leader_election.cc:343] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.557068 26524 tablet_bootstrap.cc:654] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.560082 26462 leader_election.cc:290] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.565857 26462 raft_consensus.cc:2804] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.572914 26462 raft_consensus.cc:493] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.573949 26526 tablet_bootstrap.cc:654] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.573474 26462 raft_consensus.cc:3060] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.590199 26523 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.590755 26523 ts_tablet_manager.cc:1403] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.016s
I20260709 13:07:49.590929 26526 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.591481 26526 ts_tablet_manager.cc:1403] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 13:07:49.591805 26524 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.592344 26524 ts_tablet_manager.cc:1403] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.016s
I20260709 13:07:49.594035 26523 raft_consensus.cc:359] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.594856 26523 raft_consensus.cc:385] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.594532 26526 raft_consensus.cc:359] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.595181 26523 raft_consensus.cc:740] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.595270 26526 raft_consensus.cc:385] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.595649 26526 raft_consensus.cc:740] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.595389 26524 raft_consensus.cc:359] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.596382 26524 raft_consensus.cc:385] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.596942 26524 raft_consensus.cc:740] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.596444 26526 consensus_queue.cc:260] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.596153 26523 consensus_queue.cc:260] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.597735 26524 consensus_queue.cc:260] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.598932 26526 ts_tablet_manager.cc:1434] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:49.600236 26524 ts_tablet_manager.cc:1434] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:49.600315 26526 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.601421 26524 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.606807 26526 tablet_bootstrap.cc:654] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.609264 26524 tablet_bootstrap.cc:654] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.593385 26462 raft_consensus.cc:515] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.619334 26526 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.619982 26526 ts_tablet_manager.cc:1403] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 13:07:49.623494 26462 leader_election.cc:290] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
W20260709 13:07:49.623986 26408 tablet.cc:2406] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:49.614274 26523 ts_tablet_manager.cc:1434] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:07:49.625524 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a44bfdc7b144878d22681831f1aeb9" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:49.625561 26523 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.623445 26526 raft_consensus.cc:359] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.630493 26526 raft_consensus.cc:385] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:49.630734 26196 leader_election.cc:343] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.630861 26526 raft_consensus.cc:740] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.632200 26526 consensus_queue.cc:260] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.634713 26526 ts_tablet_manager.cc:1434] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:49.635222 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a44bfdc7b144878d22681831f1aeb9" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.635782 26526 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.636157 26381 raft_consensus.cc:3060] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.641942 26523 tablet_bootstrap.cc:654] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.643724 26526 tablet_bootstrap.cc:654] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.647174 26500 raft_consensus.cc:493] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.647989 26500 raft_consensus.cc:515] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.651438 26381 raft_consensus.cc:2468] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:49.655324 26462 raft_consensus.cc:493] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.655581 26195 leader_election.cc:304] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:49.655973 26462 raft_consensus.cc:515] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.656594 26535 raft_consensus.cc:2804] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.658406 26462 leader_election.cc:290] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.658648 26523 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.659504 26523 ts_tablet_manager.cc:1403] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.004s
I20260709 13:07:49.663354 26526 tablet_bootstrap.cc:492] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.664139 26526 ts_tablet_manager.cc:1403] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.009s
I20260709 13:07:49.664184 26523 raft_consensus.cc:359] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.665460 26523 raft_consensus.cc:385] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.665879 26523 raft_consensus.cc:740] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.666906 26523 consensus_queue.cc:260] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.669250 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.668174 26526 raft_consensus.cc:359] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.669962 26526 raft_consensus.cc:385] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.670015 26382 raft_consensus.cc:2468] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:49.670470 26526 raft_consensus.cc:740] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.670540 26523 ts_tablet_manager.cc:1434] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 13:07:49.671370 26526 consensus_queue.cc:260] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.672505 26523 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.673949 26526 ts_tablet_manager.cc:1434] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:07:49.674460 26195 leader_election.cc:304] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:49.675201 26526 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.675062 26535 raft_consensus.cc:697] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:49.683104 26523 tablet_bootstrap.cc:654] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.697767 26544 raft_consensus.cc:2804] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.698246 26544 raft_consensus.cc:493] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.698597 26544 raft_consensus.cc:3060] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.705142 26526 tablet_bootstrap.cc:654] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.708725 26544 raft_consensus.cc:515] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.651793 26500 leader_election.cc:290] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.712612 26500 raft_consensus.cc:493] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.714850 26500 raft_consensus.cc:515] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.715306 26523 tablet_bootstrap.cc:492] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.716365 26523 ts_tablet_manager.cc:1403] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.019s
I20260709 13:07:49.717383 26535 consensus_queue.cc:237] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.719012 26526 tablet_bootstrap.cc:492] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.719619 26526 ts_tablet_manager.cc:1403] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.021s
I20260709 13:07:49.721056 26523 raft_consensus.cc:359] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.722266 26523 raft_consensus.cc:385] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.722699 26523 raft_consensus.cc:740] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.723320 26526 raft_consensus.cc:359] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.724193 26526 raft_consensus.cc:385] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.723578 26523 consensus_queue.cc:260] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.724505 26526 raft_consensus.cc:740] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.725296 26526 consensus_queue.cc:260] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.727814 26551 raft_consensus.cc:493] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.728344 26551 raft_consensus.cc:515] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.731184 26551 leader_election.cc:290] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.734419 26526 ts_tablet_manager.cc:1434] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:49.734465 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.735634 26526 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.738106 26523 ts_tablet_manager.cc:1434] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:07:49.739405 26523 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.745225 26526 tablet_bootstrap.cc:654] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.745302 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.746694 26231 raft_consensus.cc:2468] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:49.750233 26512 raft_consensus.cc:493] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.750833 26512 raft_consensus.cc:515] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.753754 26512 leader_election.cc:290] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:49.756211 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
W20260709 13:07:49.757513 26195 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.195:45787, user_credentials={real_user=slave}} blocked reactor thread for 51895.5us
I20260709 13:07:49.757694 26306 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:49.759315 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.765130 26346 leader_election.cc:304] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:49.765432 26231 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:49.761171 26526 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.766139 26512 raft_consensus.cc:2804] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.762576 26272 leader_election.cc:304] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:49.767064 26512 raft_consensus.cc:493] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.767376 26549 raft_consensus.cc:2804] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.767530 26512 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.767827 26549 raft_consensus.cc:493] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.768253 26549 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.761376 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.775825 26546 raft_consensus.cc:493] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.766527 26526 ts_tablet_manager.cc:1403] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.005s
I20260709 13:07:49.782320 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.783124 26382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.783414 26231 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 0.
I20260709 13:07:49.783818 26382 raft_consensus.cc:3060] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.785833 26526 raft_consensus.cc:359] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.786620 26526 raft_consensus.cc:385] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.786911 26526 raft_consensus.cc:740] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.787598 26526 consensus_queue.cc:260] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.777987 26272 leader_election.cc:304] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:49.788743 26540 raft_consensus.cc:493] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.789470 26540 raft_consensus.cc:515] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.790445 26526 ts_tablet_manager.cc:1434] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:49.791661 26526 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.792575 26540 leader_election.cc:290] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.778416 26523 tablet_bootstrap.cc:654] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.793401 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:49.793630 26546 raft_consensus.cc:515] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.794220 26306 raft_consensus.cc:2468] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:49.776476 26549 raft_consensus.cc:515] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.778600 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.792766 26382 raft_consensus.cc:2468] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:49.803287 26380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32538c59d81b4409b0b8f10cd66f9057" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.804023 26380 raft_consensus.cc:2468] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:49.805544 26195 leader_election.cc:304] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:49.807411 26550 raft_consensus.cc:2804] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.807893 26550 raft_consensus.cc:697] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:49.808758 26550 consensus_queue.cc:237] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.809218 26549 leader_election.cc:290] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
W20260709 13:07:49.811868 26271 leader_election.cc:343] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.811993 26523 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.812875 26523 ts_tablet_manager.cc:1403] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.074s	user 0.018s	sys 0.002s
I20260709 13:07:49.800901 26546 leader_election.cc:290] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.814333 26195 leader_election.cc:304] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:49.802904 26233 raft_consensus.cc:2468] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
I20260709 13:07:49.822540 26546 raft_consensus.cc:2804] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.806327 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32538c59d81b4409b0b8f10cd66f9057" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:49.822928 26546 raft_consensus.cc:493] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.823256 26308 raft_consensus.cc:2468] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:49.823817 26380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.763667 26500 leader_election.cc:290] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.818588 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.821015 26379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:49.823319 26546 raft_consensus.cc:3060] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.802402 26305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
W20260709 13:07:49.826463 26272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.13.163.193:45401, user_credentials={real_user=slave}} blocked reactor thread for 48579.4us
I20260709 13:07:49.781107 26544 leader_election.cc:290] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.824908 26512 raft_consensus.cc:515] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.817504 26523 raft_consensus.cc:359] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.828411 26305 raft_consensus.cc:3060] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.844889 26523 raft_consensus.cc:385] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.835016 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf6f070920e4604b451af2f70d02962" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.826244 26549 raft_consensus.cc:2804] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.845647 26549 raft_consensus.cc:493] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.840176 26526 tablet_bootstrap.cc:654] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:49.841627 26271 leader_election.cc:343] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.842638 26381 raft_consensus.cc:2393] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:49.844035 26512 leader_election.cc:290] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:49.844696 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:49.831493 26377 raft_consensus.cc:2393] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate e600c189c03f439abeed018164fc6d39 in this term.
I20260709 13:07:49.846710 26549 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.840168 26524 tablet_bootstrap.cc:492] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.852523 26231 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.853792 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:49.855032 26524 ts_tablet_manager.cc:1403] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.254s	user 0.102s	sys 0.094s
W20260709 13:07:49.854614 26271 leader_election.cc:343] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.859277 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:49.860052 26233 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.861105 26546 raft_consensus.cc:515] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.863332 26546 leader_election.cc:290] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.865474 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32538c59d81b4409b0b8f10cd66f9057" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.866255 26377 raft_consensus.cc:3060] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.866530 26524 raft_consensus.cc:359] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.867333 26524 raft_consensus.cc:385] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.867659 26524 raft_consensus.cc:740] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.867659 26549 raft_consensus.cc:515] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.868546 26524 consensus_queue.cc:260] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.869166 26308 raft_consensus.cc:2393] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a31357e1e0cc4058bb24dc26e0aa2522 in current term 1: Already voted for candidate e72cb39b26f7438abc76d6f96e894597 in this term.
I20260709 13:07:49.870237 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32538c59d81b4409b0b8f10cd66f9057" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:49.870867 26306 raft_consensus.cc:3060] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.871049 26524 ts_tablet_manager.cc:1434] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:07:49.872285 26305 raft_consensus.cc:2468] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:49.875294 26549 leader_election.cc:290] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.878046 26231 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 1.
I20260709 13:07:49.882154 26272 leader_election.cc:304] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:49.884158 26272 leader_election.cc:304] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39
I20260709 13:07:49.846570 26523 raft_consensus.cc:740] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.885011 26540 raft_consensus.cc:2804] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:49.885399 26540 raft_consensus.cc:493] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.885783 26540 raft_consensus.cc:3060] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.885444 26523 consensus_queue.cc:260] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.885659 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.886301 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:49.886775 26381 raft_consensus.cc:2393] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e72cb39b26f7438abc76d6f96e894597 in current term 1: Already voted for candidate a31357e1e0cc4058bb24dc26e0aa2522 in this term.
I20260709 13:07:49.887969 26523 ts_tablet_manager.cc:1434] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.003s
I20260709 13:07:49.888104 26549 raft_consensus.cc:2749] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:49.878046 26306 raft_consensus.cc:2468] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:49.889582 26346 leader_election.cc:304] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:49.891228 26512 raft_consensus.cc:2804] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.891760 26524 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.891950 26512 raft_consensus.cc:697] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:49.893500 26512 consensus_queue.cc:237] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.895545 26540 raft_consensus.cc:515] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.897089 26272 leader_election.cc:304] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39
I20260709 13:07:49.897089 26196 leader_election.cc:304] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:49.898219 26550 raft_consensus.cc:2804] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.898728 26550 raft_consensus.cc:697] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:49.899142 26377 raft_consensus.cc:2468] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:49.899464 26540 leader_election.cc:290] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:49.903800 26233 raft_consensus.cc:2468] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:49.900893 26550 consensus_queue.cc:237] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.910090 26556 raft_consensus.cc:493] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.910894 26556 raft_consensus.cc:515] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.915298 26523 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:49.915583 26232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:49.905581 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:49.905733 26231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:49.902540 26549 raft_consensus.cc:2749] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:49.919373 26231 raft_consensus.cc:3060] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:49.921797 26272 leader_election.cc:304] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:49.922586 26526 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.922676 26549 raft_consensus.cc:2804] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.923285 26526 ts_tablet_manager.cc:1403] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.011s
I20260709 13:07:49.923601 26549 raft_consensus.cc:697] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:49.924814 26549 consensus_queue.cc:237] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.926340 26526 raft_consensus.cc:359] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.927094 26526 raft_consensus.cc:385] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.927340 26526 raft_consensus.cc:740] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.931149 26526 consensus_queue.cc:260] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.932705 26524 tablet_bootstrap.cc:654] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.935509 26526 ts_tablet_manager.cc:1434] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:07:49.936661 26526 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Bootstrap starting.
I20260709 13:07:49.932406 26128 catalog_manager.cc:5697] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.938180 26128 catalog_manager.cc:5697] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.942234 26523 tablet_bootstrap.cc:654] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:49.946905 26271 leader_election.cc:343] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:49.943614 26556 leader_election.cc:290] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:49.944314 26231 raft_consensus.cc:2468] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:49.950821 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:49.952212 26272 leader_election.cc:304] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:49.953522 26549 raft_consensus.cc:2804] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:49.953570 26346 leader_election.cc:304] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522; no voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:49.954016 26526 tablet_bootstrap.cc:654] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:49.954449 26556 raft_consensus.cc:2749] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:49.954125 26549 raft_consensus.cc:697] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:49.958686 26549 consensus_queue.cc:237] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.977696 26526 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: No bootstrap required, opened a new log
I20260709 13:07:49.978448 26526 ts_tablet_manager.cc:1403] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.017s
I20260709 13:07:49.980301 26524 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:49.981015 26524 ts_tablet_manager.cc:1403] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.024s
I20260709 13:07:49.980911 26526 raft_consensus.cc:359] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.981599 26526 raft_consensus.cc:385] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.981839 26526 raft_consensus.cc:740] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.982502 26526 consensus_queue.cc:260] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.985463 26526 ts_tablet_manager.cc:1434] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:49.990465 26524 raft_consensus.cc:359] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.991317 26524 raft_consensus.cc:385] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.991632 26524 raft_consensus.cc:740] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.991829 26523 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:49.992373 26523 ts_tablet_manager.cc:1403] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.012s
I20260709 13:07:49.992421 26524 consensus_queue.cc:260] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.994884 26524 ts_tablet_manager.cc:1434] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:07:49.995265 26523 raft_consensus.cc:359] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.995950 26524 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:49.996019 26523 raft_consensus.cc:385] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:49.995564 26126 catalog_manager.cc:5697] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 0 to 1, leader changed from <none> to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.996446 26523 raft_consensus.cc:740] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:49.997339 26523 consensus_queue.cc:260] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:49.999706 26523 ts_tablet_manager.cc:1434] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:07:50.000819 26523 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Bootstrap starting.
I20260709 13:07:50.006740 26524 tablet_bootstrap.cc:654] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.008705 26127 catalog_manager.cc:5697] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 1 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.009989 26128 catalog_manager.cc:5697] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.014983 26523 tablet_bootstrap.cc:654] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.026820 26524 tablet_bootstrap.cc:492] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:50.027429 26524 ts_tablet_manager.cc:1403] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.009s
I20260709 13:07:50.030741 26524 raft_consensus.cc:359] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.031548 26524 raft_consensus.cc:385] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.031900 26524 raft_consensus.cc:740] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.033383 26549 raft_consensus.cc:493] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.033583 26524 consensus_queue.cc:260] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.033950 26549 raft_consensus.cc:515] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.036106 26524 ts_tablet_manager.cc:1434] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.004s
I20260709 13:07:50.034068 26126 catalog_manager.cc:5697] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 0 to 1, leader changed from <none> to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.037827 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:50.037827 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:50.038646 26233 raft_consensus.cc:2468] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 0.
W20260709 13:07:50.039228 26271 leader_election.cc:343] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:50.039847 26549 leader_election.cc:290] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.040064 26272 leader_election.cc:304] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:50.040555 26523 tablet_bootstrap.cc:492] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: No bootstrap required, opened a new log
I20260709 13:07:50.040980 26549 raft_consensus.cc:2804] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:50.041504 26549 raft_consensus.cc:493] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.041867 26549 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.042037 26524 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:50.041097 26523 ts_tablet_manager.cc:1403] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.010s
I20260709 13:07:50.046775 26523 raft_consensus.cc:359] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.047448 26523 raft_consensus.cc:385] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.047732 26523 raft_consensus.cc:740] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.048390 26523 consensus_queue.cc:260] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.049080 26524 tablet_bootstrap.cc:654] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.050884 26523 ts_tablet_manager.cc:1434] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:50.051570 26550 raft_consensus.cc:493] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.052070 26550 raft_consensus.cc:515] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.055576 26550 leader_election.cc:290] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.057165 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:50.057946 26377 raft_consensus.cc:2468] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:50.059216 26195 leader_election.cc:304] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:50.057168 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:50.060071 26550 raft_consensus.cc:2804] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:50.060402 26306 raft_consensus.cc:2468] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:50.060555 26550 raft_consensus.cc:493] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.062027 26550 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.062426 26549 raft_consensus.cc:515] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.064704 26524 tablet_bootstrap.cc:492] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:50.065321 26524 ts_tablet_manager.cc:1403] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.009s
I20260709 13:07:50.065778 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:50.066439 26233 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.064723 26549 leader_election.cc:290] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.068856 26524 raft_consensus.cc:359] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.069868 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.070043 26524 raft_consensus.cc:385] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.070570 26524 raft_consensus.cc:740] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.070401 26550 raft_consensus.cc:515] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
W20260709 13:07:50.071316 26271 leader_election.cc:343] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:50.071514 26524 consensus_queue.cc:260] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.073789 26524 ts_tablet_manager.cc:1434] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:50.074913 26524 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Bootstrap starting.
I20260709 13:07:50.075155 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:50.075361 26550 leader_election.cc:290] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.075800 26306 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.077512 26377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.078230 26377 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.080879 26524 tablet_bootstrap.cc:654] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.083837 26306 raft_consensus.cc:2468] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:50.084092 26512 raft_consensus.cc:493] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.084702 26512 raft_consensus.cc:515] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.087023 26512 leader_election.cc:290] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:50.087332 26377 raft_consensus.cc:2468] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:50.086861 26233 raft_consensus.cc:2468] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 1.
I20260709 13:07:50.088554 26195 leader_election.cc:304] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:50.089530 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:50.089632 26272 leader_election.cc:304] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:50.089965 26550 raft_consensus.cc:2804] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:50.090560 26549 raft_consensus.cc:2804] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:50.091100 26549 raft_consensus.cc:697] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.088542 26524 tablet_bootstrap.cc:492] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: No bootstrap required, opened a new log
I20260709 13:07:50.092239 26524 ts_tablet_manager.cc:1403] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 13:07:50.092024 26549 consensus_queue.cc:237] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.095233 26524 raft_consensus.cc:359] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.095922 26524 raft_consensus.cc:385] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.096194 26524 raft_consensus.cc:740] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.097035 26524 consensus_queue.cc:260] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.099081 26524 ts_tablet_manager.cc:1434] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 13:07:50.101814 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d3b4f5c4294330bb8f10ca2a7c80b3" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:50.102726 26233 raft_consensus.cc:2393] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a31357e1e0cc4058bb24dc26e0aa2522 in current term 1: Already voted for candidate e72cb39b26f7438abc76d6f96e894597 in this term.
I20260709 13:07:50.102365 26128 catalog_manager.cc:5697] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 0 to 1, leader changed from <none> to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 1 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.104107 26346 leader_election.cc:304] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522; no voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:50.104966 26512 raft_consensus.cc:3060] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.106108 26550 raft_consensus.cc:697] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:50.107096 26550 consensus_queue.cc:237] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.113649 26512 raft_consensus.cc:2749] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:50.116765 26128 catalog_manager.cc:5697] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.138827 26550 consensus_queue.cc:1048] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.153016 26550 raft_consensus.cc:493] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.153756 26550 raft_consensus.cc:515] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.156960 26550 leader_election.cc:290] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.157661 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:50.158394 26306 raft_consensus.cc:2468] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:50.158551 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" is_pre_election: true
I20260709 13:07:50.159272 26381 raft_consensus.cc:2468] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 0.
I20260709 13:07:50.159695 26196 leader_election.cc:304] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:50.160763 26550 raft_consensus.cc:2804] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:50.161545 26550 raft_consensus.cc:493] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.161885 26550 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.167562 26550 raft_consensus.cc:515] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.169625 26550 leader_election.cc:290] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.170668 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:50.170908 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.171404 26306 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.171620 26381 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.176023 26305 raft_consensus.cc:3060] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.179252 26306 raft_consensus.cc:2468] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:50.180440 26196 leader_election.cc:304] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:50.180673 26381 raft_consensus.cc:2468] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 1.
I20260709 13:07:50.181355 26552 raft_consensus.cc:2804] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:50.181893 26552 raft_consensus.cc:697] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:50.182685 26552 consensus_queue.cc:237] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.184667 26550 consensus_queue.cc:1048] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.192227 26128 catalog_manager.cc:5697] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 0 to 1, leader changed from <none> to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 1 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.202090 26512 raft_consensus.cc:493] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.202669 26512 raft_consensus.cc:515] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.204489 26512 leader_election.cc:290] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:50.205108 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e600c189c03f439abeed018164fc6d39" is_pre_election: true
I20260709 13:07:50.205691 26305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72cb39b26f7438abc76d6f96e894597" is_pre_election: true
I20260709 13:07:50.205915 26233 raft_consensus.cc:2393] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a31357e1e0cc4058bb24dc26e0aa2522 in current term 1: Already voted for candidate e72cb39b26f7438abc76d6f96e894597 in this term.
I20260709 13:07:50.207260 26346 leader_election.cc:304] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522; no voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:50.208063 26512 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.213444 26512 raft_consensus.cc:2749] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:50.235054 26552 consensus_queue.cc:1048] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.243681 26305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
mode: GRACEFUL
new_leader_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:34882
I20260709 13:07:50.244501 26305 raft_consensus.cc:606] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Received request to transfer leadership to TS e600c189c03f439abeed018164fc6d39
I20260709 13:07:50.249012 26544 consensus_queue.cc:1048] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.260398 26377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34602
I20260709 13:07:50.261211 26377 raft_consensus.cc:606] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:50.270418 26233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:07:50.271185 26233 raft_consensus.cc:606] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:50.276440 26233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4846873ec5e549d385583ce30767e79b"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
mode: GRACEFUL
new_leader_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:07:50.277058 26233 raft_consensus.cc:606] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Received request to transfer leadership to TS e72cb39b26f7438abc76d6f96e894597
I20260709 13:07:50.282519 26306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
mode: GRACEFUL
new_leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:34882
I20260709 13:07:50.283113 26306 raft_consensus.cc:606] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Received request to transfer leadership to TS a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:50.375383 26540 consensus_queue.cc:1048] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.387588 26583 consensus_queue.cc:1048] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.389182 26546 consensus_queue.cc:1048] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.408262 26550 consensus_queue.cc:1048] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.414440 26512 consensus_queue.cc:1048] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.443509 26556 consensus_queue.cc:1048] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.450930 26379 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.457827 26512 raft_consensus.cc:993] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.458437 26512 raft_consensus.cc:1081] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.460312 26305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34876
I20260709 13:07:50.460963 26305 raft_consensus.cc:493] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.461341 26305 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.461180 26583 consensus_queue.cc:1048] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.467747 26305 raft_consensus.cc:515] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.470388 26305 leader_election.cc:290] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.472371 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:50.473066 26233 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.476908 26583 consensus_queue.cc:1048] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.478964 26379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2842c695cdfb4ab8b762d80ec279cac2" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.479688 26379 raft_consensus.cc:3055] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.480039 26379 raft_consensus.cc:740] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:50.481212 26233 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:50.481048 26379 consensus_queue.cc:260] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.482442 26379 raft_consensus.cc:3060] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.482599 26272 leader_election.cc:304] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:50.483842 26583 raft_consensus.cc:2804] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:50.484412 26583 raft_consensus.cc:697] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.485363 26583 consensus_queue.cc:237] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.491039 26379 raft_consensus.cc:2468] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:50.493804 26583 raft_consensus.cc:993] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: : Instructing follower e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:50.494451 26549 raft_consensus.cc:1081] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Signalling peer e600c189c03f439abeed018164fc6d39 to start an election
I20260709 13:07:50.496079 26231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
 from {username='slave'} at 127.0.0.1:47450
I20260709 13:07:50.496604 26231 raft_consensus.cc:493] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.497372 26231 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.499447 26129 catalog_manager.cc:5697] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.502630 26544 consensus_queue.cc:1048] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.503823 26231 raft_consensus.cc:515] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.508589 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:50.509298 26306 raft_consensus.cc:3055] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.509677 26306 raft_consensus.cc:740] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.510579 26306 consensus_queue.cc:260] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.511946 26306 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.514191 26231 leader_election.cc:290] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Requested vote from peers e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.514353 26379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102ccba0b0614ab9bad959c1d1b3df3e" candidate_uuid: "e600c189c03f439abeed018164fc6d39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.514992 26379 raft_consensus.cc:3060] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.518733 26546 raft_consensus.cc:993] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.519253 26546 raft_consensus.cc:1081] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.524718 26306 raft_consensus.cc:2468] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:50.524718 26379 raft_consensus.cc:2468] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e600c189c03f439abeed018164fc6d39 in term 2.
I20260709 13:07:50.527868 26583 consensus_queue.cc:1048] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.528385 26195 leader_election.cc:304] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:50.529765 26546 raft_consensus.cc:2804] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:50.530361 26544 consensus_queue.cc:1048] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.530359 26546 raft_consensus.cc:697] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Becoming Leader. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:50.532656 26306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34860
I20260709 13:07:50.536978 26546 consensus_queue.cc:237] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.541525 26306 raft_consensus.cc:493] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.542065 26306 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.549456 26306 raft_consensus.cc:515] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.551614 26306 leader_election.cc:290] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.552333 26540 consensus_queue.cc:1048] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.552057 26128 catalog_manager.cc:5697] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 reported cstate change: term changed from 1 to 2, leader changed from e72cb39b26f7438abc76d6f96e894597 (127.13.163.194) to e600c189c03f439abeed018164fc6d39 (127.13.163.193). New cstate: current_term: 2 leader_uuid: "e600c189c03f439abeed018164fc6d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.552956 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:50.553627 26233 raft_consensus.cc:3055] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.553965 26233 raft_consensus.cc:740] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:50.554637 26379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0568319830424f82a86cfda26b39bb" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.554765 26233 consensus_queue.cc:260] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.555964 26233 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.563723 26233 raft_consensus.cc:2468] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:50.564599 26549 raft_consensus.cc:993] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: : Instructing follower a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:50.564983 26272 leader_election.cc:304] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:50.566028 26549 raft_consensus.cc:2804] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:50.566742 26549 raft_consensus.cc:697] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.568013 26549 consensus_queue.cc:237] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.565238 26583 raft_consensus.cc:1081] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Signalling peer a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:50.575312 26377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:34594
I20260709 13:07:50.575976 26377 raft_consensus.cc:493] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.576395 26377 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.580219 26128 catalog_manager.cc:5697] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from e600c189c03f439abeed018164fc6d39 (127.13.163.193) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.582479 26377 raft_consensus.cc:515] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.586390 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:50.587045 26233 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.588268 26306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6540caac4247bf8f6b3a36418ad473" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:50.588938 26306 raft_consensus.cc:3055] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.589269 26306 raft_consensus.cc:740] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.589973 26306 consensus_queue.cc:260] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.591025 26306 raft_consensus.cc:3060] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.594941 26233 raft_consensus.cc:2468] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 2.
I20260709 13:07:50.596194 26346 leader_election.cc:304] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:50.597090 26512 raft_consensus.cc:2804] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:50.597589 26512 raft_consensus.cc:697] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:50.586759 26377 leader_election.cc:290] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:50.598464 26512 consensus_queue.cc:237] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.602851 26306 raft_consensus.cc:2468] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 2.
I20260709 13:07:50.606762 26128 catalog_manager.cc:5697] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 1 to 2, leader changed from e72cb39b26f7438abc76d6f96e894597 (127.13.163.194) to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 2 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.639037 26550 consensus_queue.cc:1048] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:50.645622 26544 raft_consensus.cc:993] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: : Instructing follower e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.646175 26544 raft_consensus.cc:1081] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Signalling peer e72cb39b26f7438abc76d6f96e894597 to start an election
I20260709 13:07:50.648200 26305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4846873ec5e549d385583ce30767e79b"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
 from {username='slave'} at 127.0.0.1:34860
I20260709 13:07:50.649833 26305 raft_consensus.cc:493] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.650279 26305 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.652916 26550 consensus_queue.cc:1048] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:50.658133 26305 raft_consensus.cc:515] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.661305 26305 leader_election.cc:290] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195:45787)
I20260709 13:07:50.663409 26233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:50.663450 26381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4846873ec5e549d385583ce30767e79b" candidate_uuid: "e72cb39b26f7438abc76d6f96e894597" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
I20260709 13:07:50.664219 26233 raft_consensus.cc:3055] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.664603 26233 raft_consensus.cc:740] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:50.665433 26233 consensus_queue.cc:260] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.666651 26233 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.673991 26233 raft_consensus.cc:2468] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72cb39b26f7438abc76d6f96e894597 in term 2.
I20260709 13:07:50.675196 26272 leader_election.cc:304] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e600c189c03f439abeed018164fc6d39, e72cb39b26f7438abc76d6f96e894597; no voters: a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:50.676025 26583 raft_consensus.cc:2804] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:50.676635 26583 raft_consensus.cc:697] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Leader. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:50.677556 26583 consensus_queue.cc:237] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:50.685884 26128 catalog_manager.cc:5697] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 reported cstate change: term changed from 1 to 2, leader changed from e600c189c03f439abeed018164fc6d39 (127.13.163.193) to e72cb39b26f7438abc76d6f96e894597 (127.13.163.194). New cstate: current_term: 2 leader_uuid: "e72cb39b26f7438abc76d6f96e894597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.934258 26377 raft_consensus.cc:1275] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:50.935819 26583 consensus_queue.cc:1048] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:50.947485 26233 raft_consensus.cc:1275] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:50.950963 26583 consensus_queue.cc:1048] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.039572 26377 raft_consensus.cc:3060] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:51.045782 26377 raft_consensus.cc:1275] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.047566 26549 consensus_queue.cc:1048] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.056339 26231 raft_consensus.cc:1275] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.059016 26583 consensus_queue.cc:1048] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.152422 26231 raft_consensus.cc:1275] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.153393 26305 raft_consensus.cc:1275] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.153983 26512 consensus_queue.cc:1048] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:51.155879 26550 consensus_queue.cc:1048] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.166333 26305 raft_consensus.cc:1275] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.182065 26598 consensus_queue.cc:1048] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.184547 26377 raft_consensus.cc:1275] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e600c189c03f439abeed018164fc6d39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.186988 26546 consensus_queue.cc:1048] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.220029 26231 raft_consensus.cc:1275] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.221486 26583 consensus_queue.cc:1048] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:51.231724 26377 raft_consensus.cc:3060] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:51.238742 26377 raft_consensus.cc:1275] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Refusing update from remote peer e72cb39b26f7438abc76d6f96e894597: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:51.239967 26597 consensus_queue.cc:1048] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.286754 26126 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48188:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:07:53.307219 26126 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48188:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:07:53.412019 13967 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:53.443849 26643 tablet.cc:1724] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.514935 26629 log.cc:927] Time spent T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
W20260709 13:07:53.514935 26631 log.cc:927] Time spent T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
I20260709 13:07:53.481891 26649 tablet.cc:1724] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.524178 26633 log.cc:927] Time spent T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 13:07:53.544592 26657 tablet.cc:1724] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.553570 26640 log.cc:927] Time spent T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
W20260709 13:07:53.566663 26636 log.cc:927] Time spent T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 13:07:53.593011 26653 tablet.cc:1724] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.607065 26652 log.cc:927] Time spent T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:53.608108 26654 tablet.cc:1724] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.656486 26228 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5"
dest_uuid: "e600c189c03f439abeed018164fc6d39"
mode: GRACEFUL
new_leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:52910
I20260709 13:07:53.657279 26228 raft_consensus.cc:606] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Received request to transfer leadership to TS a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:53.666744 26658 tablet.cc:1724] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.583659 26655 tablet.cc:1724] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.670888 26667 tablet.cc:1724] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.671870 26662 tablet.cc:1724] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.678493 26668 tablet.cc:1724] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.696828 26657 tablet.cc:1724] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.697943 26608 raft_consensus.cc:993] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: : Instructing follower a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:53.704241 26608 raft_consensus.cc:1081] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Signalling peer a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:53.666350 26664 tablet.cc:1724] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.711133 26376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:34598
I20260709 13:07:53.703773 26655 tablet.cc:1724] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.719486 26376 raft_consensus.cc:493] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.719944 26376 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:53.716794 26657 tablet.cc:1724] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.717510 26660 tablet.cc:1724] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.731835 26272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:45401, user_credentials={real_user=slave}} blocked reactor thread for 85911.9us
I20260709 13:07:53.735774 26668 tablet.cc:1724] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.737639 26643 tablet.cc:1724] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:53.740981 26346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.13.163.193:45401, user_credentials={real_user=slave}} blocked reactor thread for 99772.6us
I20260709 13:07:53.773005 26376 raft_consensus.cc:515] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:53.774405 26664 tablet.cc:1724] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.781733 26658 tablet.cc:1724] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.789181 26654 tablet.cc:1724] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.791908 26655 tablet.cc:1724] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.791563 26653 tablet.cc:1724] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.792635 26662 tablet.cc:1724] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.804751 26376 leader_election.cc:290] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 3 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:53.806406 26230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:53.807889 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c027ecfd3b4d3e889ef85da3520df5" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:53.815868 26230 raft_consensus.cc:3055] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:53.816260 26230 raft_consensus.cc:740] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e600c189c03f439abeed018164fc6d39, State: Running, Role: LEADER
I20260709 13:07:53.784770 26667 tablet.cc:1724] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.820397 26308 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:53.817762 26230 consensus_queue.cc:260] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:53.825613 26230 raft_consensus.cc:3060] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:53.831969 26308 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 3.
I20260709 13:07:53.833624 26346 leader_election.cc:304] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e72cb39b26f7438abc76d6f96e894597; no voters: 
I20260709 13:07:53.834682 26665 raft_consensus.cc:2804] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:53.835179 26665 raft_consensus.cc:697] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:53.835979 26665 consensus_queue.cc:237] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:53.818245 26643 tablet.cc:1724] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.810109 26660 tablet.cc:1724] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.851262 26230 raft_consensus.cc:2468] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 3.
I20260709 13:07:53.855635 26308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b"
dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
mode: GRACEFUL
new_leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:38490
I20260709 13:07:53.856343 26308 raft_consensus.cc:606] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Received request to transfer leadership to TS a31357e1e0cc4058bb24dc26e0aa2522
I20260709 13:07:53.855700 26664 tablet.cc:1724] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:53.857914 13967 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:53.858601 13967 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:53.859639 26129 catalog_manager.cc:5697] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 2 to 3, leader changed from e600c189c03f439abeed018164fc6d39 (127.13.163.193) to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.883523 26126 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: soft_deleted_table [id=b3d7bcc8e87a4e9baac9e52eddf67f48] alter complete (version 1)
I20260709 13:07:54.255119 26597 raft_consensus.cc:993] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: : Instructing follower a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:54.255657 26672 raft_consensus.cc:1081] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Signalling peer a31357e1e0cc4058bb24dc26e0aa2522 to start an election
I20260709 13:07:54.257285 26378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b"
dest_uuid: "a31357e1e0cc4058bb24dc26e0aa2522"
 from {username='slave'} at 127.0.0.1:34594
I20260709 13:07:54.257771 26378 raft_consensus.cc:493] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:54.258100 26378 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.263197 26378 raft_consensus.cc:515] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:54.264770 26378 leader_election.cc:290] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 3 election: Requested vote from peers e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401), e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625)
I20260709 13:07:54.265794 26230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e600c189c03f439abeed018164fc6d39"
I20260709 13:07:54.266083 26308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95a57fd9a10946479d74f32616f5437b" candidate_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e72cb39b26f7438abc76d6f96e894597"
I20260709 13:07:54.266499 26230 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.266714 26308 raft_consensus.cc:3055] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:54.267127 26308 raft_consensus.cc:740] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e72cb39b26f7438abc76d6f96e894597, State: Running, Role: LEADER
I20260709 13:07:54.267863 26308 consensus_queue.cc:260] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:54.268968 26308 raft_consensus.cc:3060] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.274616 26230 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 3.
I20260709 13:07:54.276291 26308 raft_consensus.cc:2468] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31357e1e0cc4058bb24dc26e0aa2522 in term 3.
I20260709 13:07:54.276160 26346 leader_election.cc:304] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31357e1e0cc4058bb24dc26e0aa2522, e600c189c03f439abeed018164fc6d39; no voters: 
I20260709 13:07:54.277799 26665 raft_consensus.cc:2804] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:54.278343 26665 raft_consensus.cc:697] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 LEADER]: Becoming Leader. State: Replica: a31357e1e0cc4058bb24dc26e0aa2522, State: Running, Role: LEADER
I20260709 13:07:54.279260 26665 consensus_queue.cc:237] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } }
I20260709 13:07:54.283334 26308 raft_consensus.cc:1275] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:54.284789 26670 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:54.289767 26128 catalog_manager.cc:5697] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 reported cstate change: term changed from 2 to 3, leader changed from e72cb39b26f7438abc76d6f96e894597 (127.13.163.194) to a31357e1e0cc4058bb24dc26e0aa2522 (127.13.163.195). New cstate: current_term: 3 leader_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31357e1e0cc4058bb24dc26e0aa2522" member_type: VOTER last_known_addr { host: "127.13.163.195" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.298813 26229 raft_consensus.cc:1275] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:54.302073 26681 consensus_queue.cc:1048] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:54.725701 26308 raft_consensus.cc:1275] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:54.728242 26681 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72cb39b26f7438abc76d6f96e894597" member_type: VOTER last_known_addr { host: "127.13.163.194" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:54.752316 26230 raft_consensus.cc:1275] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Refusing update from remote peer a31357e1e0cc4058bb24dc26e0aa2522: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:54.755957 26687 consensus_queue.cc:1048] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e600c189c03f439abeed018164fc6d39" member_type: VOTER last_known_addr { host: "127.13.163.193" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:54.928004 13967 tablet_server.cc:179] TabletServer@127.13.163.193:0 shutting down...
I20260709 13:07:54.967422 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:54.968607 13967 tablet_replica.cc:333] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.969853 13967 raft_consensus.cc:2243] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.970779 13967 raft_consensus.cc:2272] T 4846873ec5e549d385583ce30767e79b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.974025 13967 tablet_replica.cc:333] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.974858 13967 raft_consensus.cc:2243] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.975500 13967 raft_consensus.cc:2272] T 5c0568319830424f82a86cfda26b39bb P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.978502 13967 tablet_replica.cc:333] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.979288 13967 raft_consensus.cc:2243] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.979846 13967 raft_consensus.cc:2272] T 2842c695cdfb4ab8b762d80ec279cac2 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.984279 13967 tablet_replica.cc:333] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.985325 13967 raft_consensus.cc:2243] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.986128 13967 raft_consensus.cc:2272] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.989044 13967 tablet_replica.cc:333] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.989768 13967 raft_consensus.cc:2243] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.990797 13967 raft_consensus.cc:2272] T 32538c59d81b4409b0b8f10cd66f9057 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.994019 13967 tablet_replica.cc:333] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.994735 13967 raft_consensus.cc:2243] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.995702 13967 raft_consensus.cc:2272] T caf6f070920e4604b451af2f70d02962 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.998371 13967 tablet_replica.cc:333] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:54.999049 13967 raft_consensus.cc:2243] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.999775 13967 raft_consensus.cc:2272] T 04c9919a91214947966bdec9eb37aec4 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.002439 13967 tablet_replica.cc:333] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.003124 13967 raft_consensus.cc:2243] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.004108 13967 raft_consensus.cc:2272] T 33a44bfdc7b144878d22681831f1aeb9 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.007606 13967 tablet_replica.cc:333] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.008311 13967 raft_consensus.cc:2243] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.008937 13967 raft_consensus.cc:2272] T 37d87c825e49472bb270be7e0f277609 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.011777 13967 tablet_replica.cc:333] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.012485 13967 raft_consensus.cc:2243] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.013130 13967 raft_consensus.cc:2272] T 43c027ecfd3b4d3e889ef85da3520df5 P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.016652 13967 tablet_replica.cc:333] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.017519 13967 raft_consensus.cc:2243] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.018522 13967 raft_consensus.cc:2272] T c0b03910d4d549828b82cf0ea4fe938b P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.021162 13967 tablet_replica.cc:333] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.021793 13967 raft_consensus.cc:2243] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.022354 13967 raft_consensus.cc:2272] T ac6540caac4247bf8f6b3a36418ad473 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.025246 13967 tablet_replica.cc:333] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.026073 13967 raft_consensus.cc:2243] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.026942 13967 raft_consensus.cc:2272] T cae45edb272b44a9aaaa37818f4c2ce7 P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.027475 26272 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 13:07:55.031754 13967 tablet_replica.cc:333] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39: stopping tablet replica
W20260709 13:07:55.032346 26272 consensus_peers.cc:597] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.032814 13967 raft_consensus.cc:2243] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.033541 13967 raft_consensus.cc:2272] T 95a57fd9a10946479d74f32616f5437b P e600c189c03f439abeed018164fc6d39 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.036710 13967 tablet_replica.cc:333] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.037460 13967 raft_consensus.cc:2243] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.038417 13967 raft_consensus.cc:2272] T 102ccba0b0614ab9bad959c1d1b3df3e P e600c189c03f439abeed018164fc6d39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.041026 13967 tablet_replica.cc:333] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.041635 13967 raft_consensus.cc:2243] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.042663 13967 raft_consensus.cc:2272] T d54584a4053b41c99b53dda055ba490d P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.045300 13967 tablet_replica.cc:333] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.045872 13967 raft_consensus.cc:2243] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.046393 13967 raft_consensus.cc:2272] T 8508c52406f8476ba4da651c8b8b7d37 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.048763 13967 tablet_replica.cc:333] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39: stopping tablet replica
I20260709 13:07:55.049332 13967 raft_consensus.cc:2243] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.050269 13967 raft_consensus.cc:2272] T 490b6ffef368438591943bc51cc81736 P e600c189c03f439abeed018164fc6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.098681 13967 tablet_server.cc:196] TabletServer@127.13.163.193:0 shutdown complete.
I20260709 13:07:55.148171 13967 tablet_server.cc:179] TabletServer@127.13.163.194:0 shutting down...
W20260709 13:07:55.151101 26346 consensus_peers.cc:597] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.155066 26346 consensus_peers.cc:597] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.166647 26346 consensus_peers.cc:597] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.179934 26272 consensus_peers.cc:597] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.192272 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:55.193321 13967 tablet_replica.cc:333] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.194589 13967 raft_consensus.cc:2243] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.195981 13967 raft_consensus.cc:2272] T 4846873ec5e549d385583ce30767e79b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.198040 26346 consensus_peers.cc:597] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Couldn't send request to peer e72cb39b26f7438abc76d6f96e894597. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.199369 13967 tablet_replica.cc:333] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.200436 13967 raft_consensus.cc:2243] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.201144 13967 raft_consensus.cc:2272] T caf6f070920e4604b451af2f70d02962 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.203698 13967 tablet_replica.cc:333] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.204311 13967 raft_consensus.cc:2243] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.205341 13967 raft_consensus.cc:2272] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.207849 13967 tablet_replica.cc:333] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.208467 13967 raft_consensus.cc:2243] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.209437 13967 raft_consensus.cc:2272] T 5c0568319830424f82a86cfda26b39bb P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.211886 13967 tablet_replica.cc:333] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.212755 13967 raft_consensus.cc:2243] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.214111 13967 raft_consensus.cc:2272] T 2842c695cdfb4ab8b762d80ec279cac2 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.216946 13967 tablet_replica.cc:333] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.217552 13967 raft_consensus.cc:2243] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.218109 13967 raft_consensus.cc:2272] T 32538c59d81b4409b0b8f10cd66f9057 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.220767 13967 tablet_replica.cc:333] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.221484 13967 raft_consensus.cc:2243] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.222100 13967 raft_consensus.cc:2272] T 43c027ecfd3b4d3e889ef85da3520df5 P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.225893 13967 tablet_replica.cc:333] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.226825 13967 raft_consensus.cc:2243] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.227528 13967 raft_consensus.cc:2272] T 33a44bfdc7b144878d22681831f1aeb9 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.230872 26346 consensus_peers.cc:597] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Couldn't send request to peer e72cb39b26f7438abc76d6f96e894597. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.230898 13967 tablet_replica.cc:333] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.231946 13967 raft_consensus.cc:2243] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.232538 13967 raft_consensus.cc:2272] T 37d87c825e49472bb270be7e0f277609 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.235164 13967 tablet_replica.cc:333] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.235884 13967 raft_consensus.cc:2243] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.236503 13967 raft_consensus.cc:2272] T c0b03910d4d549828b82cf0ea4fe938b P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.239037 13967 tablet_replica.cc:333] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.239792 13967 raft_consensus.cc:2243] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.240885 13967 raft_consensus.cc:2272] T 04c9919a91214947966bdec9eb37aec4 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.243559 13967 tablet_replica.cc:333] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.244205 13967 raft_consensus.cc:2243] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.244726 13967 raft_consensus.cc:2272] T d54584a4053b41c99b53dda055ba490d P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.247416 13967 tablet_replica.cc:333] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.248186 13967 raft_consensus.cc:2243] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.248951 13967 raft_consensus.cc:2272] T 102ccba0b0614ab9bad959c1d1b3df3e P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.253067 13967 tablet_replica.cc:333] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.254058 13967 raft_consensus.cc:2243] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.254649 13967 raft_consensus.cc:2272] T 95a57fd9a10946479d74f32616f5437b P e72cb39b26f7438abc76d6f96e894597 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.255926 26272 consensus_peers.cc:597] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.258551 13967 tablet_replica.cc:333] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.259460 13967 raft_consensus.cc:2243] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:07:55.259618 26346 consensus_peers.cc:597] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Couldn't send request to peer e72cb39b26f7438abc76d6f96e894597. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.260648 13967 raft_consensus.cc:2272] T 8508c52406f8476ba4da651c8b8b7d37 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.263324 13967 tablet_replica.cc:333] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.264142 13967 raft_consensus.cc:2243] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.264977 13967 raft_consensus.cc:2272] T ac6540caac4247bf8f6b3a36418ad473 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.267243 13967 tablet_replica.cc:333] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.267735 13967 raft_consensus.cc:2243] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.268587 13967 raft_consensus.cc:2272] T cae45edb272b44a9aaaa37818f4c2ce7 P e72cb39b26f7438abc76d6f96e894597 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.271248 13967 tablet_replica.cc:333] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597: stopping tablet replica
I20260709 13:07:55.271776 13967 raft_consensus.cc:2243] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.272286 13967 raft_consensus.cc:2272] T 490b6ffef368438591943bc51cc81736 P e72cb39b26f7438abc76d6f96e894597 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:55.310164 26346 consensus_peers.cc:597] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e600c189c03f439abeed018164fc6d39 (127.13.163.193:45401): Couldn't send request to peer e600c189c03f439abeed018164fc6d39. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.193:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.321067 13967 tablet_server.cc:196] TabletServer@127.13.163.194:0 shutdown complete.
W20260709 13:07:55.365489 26346 consensus_peers.cc:597] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 -> Peer e72cb39b26f7438abc76d6f96e894597 (127.13.163.194:40625): Couldn't send request to peer e72cb39b26f7438abc76d6f96e894597. Status: Network error: Client connection negotiation failed: client connection to 127.13.163.194:40625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.365878 13967 tablet_server.cc:179] TabletServer@127.13.163.195:0 shutting down...
I20260709 13:07:55.399909 13967 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:55.400899 13967 tablet_replica.cc:333] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.401702 13967 raft_consensus.cc:2243] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.402282 13967 raft_consensus.cc:2272] T 4846873ec5e549d385583ce30767e79b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.404726 13967 tablet_replica.cc:333] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.405507 13967 raft_consensus.cc:2243] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.406045 13967 raft_consensus.cc:2272] T 52d3b4f5c4294330bb8f10ca2a7c80b3 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.408280 13967 tablet_replica.cc:333] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.408952 13967 raft_consensus.cc:2243] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.409487 13967 raft_consensus.cc:2272] T caf6f070920e4604b451af2f70d02962 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.411862 13967 tablet_replica.cc:333] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.412417 13967 raft_consensus.cc:2243] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.413010 13967 raft_consensus.cc:2272] T 5c0568319830424f82a86cfda26b39bb P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.415169 13967 tablet_replica.cc:333] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.415799 13967 raft_consensus.cc:2243] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.416340 13967 raft_consensus.cc:2272] T 2842c695cdfb4ab8b762d80ec279cac2 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.418573 13967 tablet_replica.cc:333] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.419102 13967 raft_consensus.cc:2243] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.419596 13967 raft_consensus.cc:2272] T 32538c59d81b4409b0b8f10cd66f9057 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.422104 13967 tablet_replica.cc:333] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.422729 13967 raft_consensus.cc:2243] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.423260 13967 raft_consensus.cc:2272] T 490b6ffef368438591943bc51cc81736 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.425973 13967 tablet_replica.cc:333] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.426682 13967 raft_consensus.cc:2243] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.427241 13967 raft_consensus.cc:2272] T c0b03910d4d549828b82cf0ea4fe938b P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.429803 13967 tablet_replica.cc:333] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.430487 13967 raft_consensus.cc:2243] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.431603 13967 raft_consensus.cc:2272] T 37d87c825e49472bb270be7e0f277609 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.434178 13967 tablet_replica.cc:333] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.434809 13967 raft_consensus.cc:2243] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.435334 13967 raft_consensus.cc:2272] T 33a44bfdc7b144878d22681831f1aeb9 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.437649 13967 tablet_replica.cc:333] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.438280 13967 raft_consensus.cc:2243] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.439234 13967 raft_consensus.cc:2272] T ac6540caac4247bf8f6b3a36418ad473 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.441643 13967 tablet_replica.cc:333] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.442239 13967 raft_consensus.cc:2243] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.442740 13967 raft_consensus.cc:2272] T cae45edb272b44a9aaaa37818f4c2ce7 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.445046 13967 tablet_replica.cc:333] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.445650 13967 raft_consensus.cc:2243] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.446214 13967 raft_consensus.cc:2272] T 102ccba0b0614ab9bad959c1d1b3df3e P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.448431 13967 tablet_replica.cc:333] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.449038 13967 raft_consensus.cc:2243] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.449563 13967 raft_consensus.cc:2272] T d54584a4053b41c99b53dda055ba490d P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.452032 13967 tablet_replica.cc:333] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.452579 13967 raft_consensus.cc:2243] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.453491 13967 raft_consensus.cc:2272] T 95a57fd9a10946479d74f32616f5437b P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.456971 13967 tablet_replica.cc:333] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.457736 13967 raft_consensus.cc:2243] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.458330 13967 raft_consensus.cc:2272] T 8508c52406f8476ba4da651c8b8b7d37 P a31357e1e0cc4058bb24dc26e0aa2522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.460747 13967 tablet_replica.cc:333] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.461514 13967 raft_consensus.cc:2243] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.462126 13967 raft_consensus.cc:2272] T 04c9919a91214947966bdec9eb37aec4 P a31357e1e0cc4058bb24dc26e0aa2522 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.464409 13967 tablet_replica.cc:333] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522: stopping tablet replica
I20260709 13:07:55.465054 13967 raft_consensus.cc:2243] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.466039 13967 raft_consensus.cc:2272] T 43c027ecfd3b4d3e889ef85da3520df5 P a31357e1e0cc4058bb24dc26e0aa2522 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.512571 13967 tablet_server.cc:196] TabletServer@127.13.163.195:0 shutdown complete.
I20260709 13:07:55.551995 13967 master.cc:562] Master@127.13.163.254:44849 shutting down...
I20260709 13:07:55.579674 13967 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:55.580334 13967 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.580686 13967 tablet_replica.cc:333] T 00000000000000000000000000000000 P 602f3b8e5a9842c1bf5ff513d83246e3: stopping tablet replica
I20260709 13:07:55.603608 13967 master.cc:584] Master@127.13.163.254:44849 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11433 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24375 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (383058 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:07:55.667019 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 411 messages since previous log ~14 seconds ago
I20260709 13:07:55.667243 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 411 messages since previous log ~14 seconds ago
I20260709 13:07:55.667403 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 408 messages since previous log ~14 seconds ago
I20260709 13:07:55.667577 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:07:55.667740 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:07:55.667915 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 1 messages since previous log ~226 seconds ago
I20260709 13:07:55.668066 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~140 seconds ago
I20260709 13:07:55.668216 13967 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 7 messages since previous log ~285 seconds ago
I20260709 13:07:55.668372 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 5 messages since previous log ~284 seconds ago
I20260709 13:07:55.668589 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 10 messages since previous log ~0 seconds ago
I20260709 13:07:55.668928 13967 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
